-- Logs begin at Sat 2022-09-24 13:45:52 UTC, end at Sat 2022-09-24 14:52:29 UTC. -- Sep 24 14:11:35.425674 dv-cinder-859193-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 24 14:11:35.432956 dv-cinder-859193-1-iscsi devstack@keystone.service[81483]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 24 14:11:35.436049 dv-cinder-859193-1-iscsi devstack@keystone.service[81483]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Sep 24 14:11:35 2022] *** Sep 24 14:11:35.436049 dv-cinder-859193-1-iscsi devstack@keystone.service[81483]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 24 14:11:35.436049 dv-cinder-859193-1-iscsi devstack@keystone.service[81483]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 24 14:11:35.436049 dv-cinder-859193-1-iscsi devstack@keystone.service[81483]: nodename: dv-cinder-859193-1-iscsi Sep 24 14:11:35.436049 dv-cinder-859193-1-iscsi devstack@keystone.service[81483]: machine: x86_64 Sep 24 14:11:35.436049 dv-cinder-859193-1-iscsi devstack@keystone.service[81483]: clock source: unix Sep 24 14:11:35.436049 dv-cinder-859193-1-iscsi devstack@keystone.service[81483]: pcre jit disabled Sep 24 14:11:35.436049 dv-cinder-859193-1-iscsi devstack@keystone.service[81483]: detected number of CPU cores: 4 Sep 24 14:11:35.436049 dv-cinder-859193-1-iscsi devstack@keystone.service[81483]: current working directory: / Sep 24 14:11:35.436049 dv-cinder-859193-1-iscsi devstack@keystone.service[81483]: detected binary path: /usr/bin/uwsgi-core Sep 24 14:11:35.436590 dv-cinder-859193-1-iscsi devstack@keystone.service[81483]: your processes number limit is 47680 Sep 24 14:11:35.436590 dv-cinder-859193-1-iscsi devstack@keystone.service[81483]: your memory page size is 4096 bytes Sep 24 14:11:35.436590 dv-cinder-859193-1-iscsi devstack@keystone.service[81483]: detected max file descriptor number: 2048 Sep 24 14:11:35.436590 dv-cinder-859193-1-iscsi devstack@keystone.service[81483]: lock engine: pthread robust mutexes Sep 24 14:11:35.436590 dv-cinder-859193-1-iscsi devstack@keystone.service[81483]: thunder lock: enabled Sep 24 14:11:35.436590 dv-cinder-859193-1-iscsi devstack@keystone.service[81483]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 24 14:11:35.436590 dv-cinder-859193-1-iscsi devstack@keystone.service[81483]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 24 14:11:35.476673 dv-cinder-859193-1-iscsi devstack@keystone.service[81483]: Python main interpreter initialized at 0x5604dffdd9c0 Sep 24 14:11:35.476673 dv-cinder-859193-1-iscsi devstack@keystone.service[81483]: python threads support enabled Sep 24 14:11:35.476673 dv-cinder-859193-1-iscsi devstack@keystone.service[81483]: your server socket listen backlog is limited to 100 connections Sep 24 14:11:35.476673 dv-cinder-859193-1-iscsi devstack@keystone.service[81483]: your mercy for graceful operations on workers is 90 seconds Sep 24 14:11:35.477050 dv-cinder-859193-1-iscsi devstack@keystone.service[81483]: mapped 403077 bytes (393 KB) for 2 cores Sep 24 14:11:35.477128 dv-cinder-859193-1-iscsi devstack@keystone.service[81483]: *** Operational MODE: preforking *** Sep 24 14:11:35.477370 dv-cinder-859193-1-iscsi devstack@keystone.service[81483]: *** uWSGI is running in multiple interpreter mode *** Sep 24 14:11:35.477370 dv-cinder-859193-1-iscsi devstack@keystone.service[81483]: spawned uWSGI master process (pid: 81483) Sep 24 14:11:35.477576 dv-cinder-859193-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 24 14:11:35.478968 dv-cinder-859193-1-iscsi devstack@keystone.service[81483]: spawned uWSGI worker 1 (pid: 81496, cores: 1) Sep 24 14:11:35.478968 dv-cinder-859193-1-iscsi devstack@keystone.service[81483]: spawned uWSGI worker 2 (pid: 81497, cores: 1) Sep 24 14:11:35.478968 dv-cinder-859193-1-iscsi devstack@keystone.service[81483]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 24 14:11:39.742678 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 24 14:11:39.742678 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: from cryptography.utils import int_from_bytes Sep 24 14:11:39.743979 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 24 14:11:39.743979 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: from cryptography.utils import int_from_bytes Sep 24 14:11:39.744102 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 24 14:11:39.744102 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: from cryptography.utils import int_from_bytes Sep 24 14:11:39.744102 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 24 14:11:39.744102 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: from cryptography.utils import int_from_bytes Sep 24 14:11:39.793329 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 24 14:11:39.793494 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 24 14:11:39.793585 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 24 14:11:39.793675 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 24 14:11:39.793755 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] command line args: [] {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 24 14:11:39.793828 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] command line args: [] {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 24 14:11:39.793958 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 24 14:11:39.794078 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 24 14:11:39.794159 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ================================================================================ {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 24 14:11:39.794264 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ================================================================================ {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 24 14:11:39.794379 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] admin_token = **** {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.794522 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] admin_token = **** {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.794613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] config_dir = [] {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.794771 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] config_dir = [] {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.794880 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.794985 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.795071 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] config_source = [] {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.795149 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] debug = True {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.795231 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] config_source = [] {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.795338 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] debug = True {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.795417 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.796678 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.796819 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.796947 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.797032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.797158 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.797243 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.797317 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.797446 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.797531 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.797604 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] list_limit = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.797670 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] list_limit = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.797744 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] log_config_append = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.797822 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] log_config_append = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.797948 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.798033 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.798111 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] log_dir = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.798234 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] log_dir = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.798317 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] log_file = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.798396 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] log_file = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.798522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.798606 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.798684 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.798806 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.798951 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.799068 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.799153 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.799234 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.799353 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.799436 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.799509 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.799639 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.799724 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.799802 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.799925 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.800049 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.800142 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.800228 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.800300 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.800423 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.800499 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.800621 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.800697 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.800829 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.800912 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.800994 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.801071 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.801194 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.801275 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.801356 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.801480 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.801558 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.801682 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] publish_errors = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.801759 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] publish_errors = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.801829 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.801943 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.802018 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.802098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.802174 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.802246 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.802315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.802405 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.802488 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.802560 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.802630 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] standard_threads = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.802705 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] standard_threads = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.802776 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.802878 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.802952 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.803022 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.803092 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.803201 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.803287 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] use_journal = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.803424 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] use_journal = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.803505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] use_json = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.803578 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] use_json = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.803654 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] use_stderr = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.803727 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] use_stderr = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.803796 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] use_syslog = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.803874 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] use_syslog = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.803952 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.804085 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 14:11:39.804201 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.804341 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.804443 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.804583 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.804702 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.804844 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.804927 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.805085 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.805163 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.805849 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.805936 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.806094 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.806172 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.806348 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.806432 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.806588 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.806673 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.806857 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.806940 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.807282 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.807474 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.807601 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.807680 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.807750 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.807821 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.807905 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.807979 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.808065 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.808144 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.808247 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.808334 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.808483 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.808567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.808724 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.808811 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.808977 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.809055 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.809222 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.809300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.809448 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.809523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.809690 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.809767 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.809945 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.810032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.810196 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.810282 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.810444 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.810523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.810676 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.810764 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.810934 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.811012 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.811165 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.811242 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.811395 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.811488 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.811633 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.811711 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.811878 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.811961 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.812110 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.812197 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.812340 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.812418 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.812569 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.812645 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.812818 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.812909 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.813052 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.813136 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.813292 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.813369 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.813520 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.813611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.813754 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.813832 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.813987 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.814064 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.814214 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.814300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.814444 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.814520 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.814670 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.814746 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.814922 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.815077 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.815142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.815199 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.815381 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.815452 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.815607 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.815684 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.815843 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.815911 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.816072 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.816148 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.816302 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.816378 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.816536 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.816603 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.816769 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.816836 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.816999 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.817078 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.817236 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.817310 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.817470 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.817538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.817776 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.817863 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.817932 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.817990 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.818150 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.818229 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.818379 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.818454 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.818608 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.818675 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.818881 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.818967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.819104 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.819180 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.819335 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.819412 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.819568 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.819644 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.819815 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.819896 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.820287 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.820357 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.820422 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.820482 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.820545 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.820604 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.820721 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.820791 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.821044 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.821715 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.821862 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.821922 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.822018 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.822090 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.822257 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.822395 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.822531 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.822651 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.822783 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.822964 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.823092 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.823228 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.823353 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.823506 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.823654 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.823806 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.823927 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.824052 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.824175 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.824336 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.824472 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.824599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.824725 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.824850 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.824974 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.825098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.825231 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.825357 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.825488 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.825612 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.825737 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.825860 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.825860 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.826112 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.826112 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.826351 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.826476 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.826602 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.826736 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.826891 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.827015 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.827140 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.827265 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.827399 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.827604 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.827733 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.827873 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.827999 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.828125 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.828259 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.828393 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.828529 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.828655 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.828780 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.828906 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.829031 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.829164 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.829298 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.829382 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.829443 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.829502 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.829562 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.829620 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.829689 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.829751 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.829809 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.829866 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.829924 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.829981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.830053 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.830115 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.830174 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.830232 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.830346 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.830444 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] auth.external = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.830513 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.830575 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.830640 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] auth.external = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.830701 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.830759 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.830963 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.831033 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.831196 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] auth.password = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.831277 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.831439 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] auth.token = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.831515 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] auth.password = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.831721 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] auth.token = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.832096 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.832096 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.832342 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.832342 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.832581 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.832581 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.832817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.832941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.833079 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.833208 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.833334 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.833460 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.833594 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.833731 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] credential.caching = True {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.833866 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.833992 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.834119 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] credential.caching = True {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.834245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.834378 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.834505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.834637 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.834768 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.835390 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.835470 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.835543 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.835634 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.835710 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.835781 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.835853 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.835924 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.835995 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.836074 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.836149 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.836221 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.836300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.836300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.836431 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.836431 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.836577 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.836737 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.836821 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.836895 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.836967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.837047 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.837133 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.837207 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] federation.caching = True {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.837279 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.837350 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.837421 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] federation.caching = True {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.837523 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.837606 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.837783 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.837872 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.837998 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.838077 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.838236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.838317 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.838471 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.838551 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.838712 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.838799 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.838978 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.839066 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.839220 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.839299 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.839455 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.839547 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.839698 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.839784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.839930 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] identity.caching = True {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.840010 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.840170 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] identity.caching = True {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.840332 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.840459 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.840533 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.840639 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.840721 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.840874 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.840963 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.841107 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.841187 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.841337 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.841416 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.841572 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.841659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.841811 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.841892 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.842056 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.842134 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.842285 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.842374 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.842516 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.842598 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.842744 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.842850 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.842998 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.843096 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.843245 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.843326 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.843490 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.843569 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.843736 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.843823 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.843982 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.844062 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.844231 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.844352 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.844468 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.844592 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.844712 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.844828 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.844944 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.845059 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.845175 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.845300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.845417 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.845534 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.845649 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.845766 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.845881 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.846006 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.846124 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.846240 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.846356 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.846472 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.846587 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.846712 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.846850 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.846970 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.847086 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.847202 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.847318 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.847448 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.847566 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.847683 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.847800 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.847924 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.848041 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.848167 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.848285 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.848402 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.848518 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.848641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.848772 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.848893 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.849010 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.849126 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.849243 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.849359 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.849485 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.849603 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.849720 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.849836 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.849954 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.850070 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.850070 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.850300 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.850417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.850576 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.850692 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.850808 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.850994 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.851104 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.851104 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.851298 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.851414 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.851530 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.851656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.851774 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.851890 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.852016 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.852134 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.user = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.852251 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.852375 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.852493 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.user = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.852608 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.852724 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.852840 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.852955 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.853081 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.853198 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.853314 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.853429 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.853544 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.853660 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.853784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.853902 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.854018 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.854134 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.854257 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.854374 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.854498 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.854616 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.854732 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.854861 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.854979 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.855106 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.855233 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.855351 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.855466 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.855581 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.855704 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.855822 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.855946 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.856064 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.856181 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.856296 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.856412 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.856527 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.856652 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.856769 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.856886 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.857001 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.857116 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.857233 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.857358 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.857475 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.857591 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.857707 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.857821 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.857937 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.858061 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.858179 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.858285 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.858400 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.858515 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.858631 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.858758 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.858898 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.859014 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.859130 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.859245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.859362 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.859490 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.859608 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.859724 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.859839 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.859954 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.860069 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.860194 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.860311 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.860428 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.860577 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.860697 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.860812 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.860938 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] resource.caching = True {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.861056 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.861176 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.861292 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] resource.caching = True {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.861416 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.861533 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.861659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.861776 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.861893 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.862008 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.862123 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.862239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.862364 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.862482 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.862598 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.862714 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.862847 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.862965 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.863164 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.863284 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.863408 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] role.caching = True {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.863527 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.863644 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] role.driver = sql {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.863773 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] role.caching = True {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.863891 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.864008 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] role.driver = sql {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.864123 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.864239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.864360 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.864487 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.864606 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.864728 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.864845 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.864962 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.865078 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.865204 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.865322 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.865439 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.865555 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.865672 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.865789 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.865914 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.866033 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.866148 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.866264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.866379 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.866495 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.866621 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.866740 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.866911 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.867029 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.867146 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.867263 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.867392 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.867512 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.867628 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.867745 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.868342 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.868460 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.868586 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.868705 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.868823 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.868939 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.869056 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.869172 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.869297 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.869416 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.869539 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.869656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.869773 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.869899 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.869899 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.870118 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.870235 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.870351 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.870467 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.870630 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.870751 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.870882 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.870882 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.871100 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.871231 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.871358 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.871358 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.871578 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.871695 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.871811 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.871928 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.872052 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.872170 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] token.caching = True {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.872286 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.872402 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.872519 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.872642 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.872768 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] token.caching = True {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.872886 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.873003 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.873118 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.873234 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.873350 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.873475 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.873594 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.873710 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.873826 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.873942 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.874058 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.874182 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.874300 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.874416 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.874532 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.874648 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.874763 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.874921 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.875041 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.875157 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.875272 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.875388 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.875504 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.875629 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.875747 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.875877 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.876004 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.876124 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.876240 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.876357 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.876486 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] database.connection = **** {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.876614 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.876741 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.876886 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.877006 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.877123 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] database.connection = **** {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.877239 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.877356 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.877482 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.877600 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.877716 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.877832 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.877948 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.878064 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.878190 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.878309 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.878425 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.878542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.878657 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.878774 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.878923 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.879042 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.879159 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.879277 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.879394 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.879510 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.879636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.879755 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.879871 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.879987 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.880104 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.880220 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.880346 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.880465 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.880581 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.880731 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.880850 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.880966 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.881092 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81496) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 24 14:11:39.881217 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.881217 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 14:11:39.881217 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81497) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 24 14:11:39.910721 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:39.911007 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:39.911174 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:39.913182 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:39.913414 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:39.913642 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:39.920943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:39.922605 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:39.949999 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:39.950222 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:39.951497 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:39.951731 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.002992 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.003498 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.051968 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.052199 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.052850 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.053087 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.208624 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.208810 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.208950 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.209030 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.209128 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.209204 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.209276 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.209347 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.209449 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.209531 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.209663 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.209747 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.233461 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.233693 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.233920 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.234142 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.234357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.234568 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.234788 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.235645 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.235880 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.236104 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.236323 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.236546 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.236761 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.236973 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.246712 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.248604 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.259503 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.259681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.259770 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.259907 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.259994 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.260132 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.260217 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.260353 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.260447 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.260567 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.260704 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.260784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81496) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.260859 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.261006 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81497) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 14:11:40.278201 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.278414 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 24 14:11:40.278711 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.278795 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.278948 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 24 14:11:40.279044 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 24 14:11:40.279229 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.279312 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.279458 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 24 14:11:40.279540 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.279732 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.279816 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.279987 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.280070 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.280242 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.280498 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.289526 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.290411 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.290656 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.290745 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.290979 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.291078 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.291253 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.291539 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.297895 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81496) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:11:40.298498 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81497) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:11:40.300791 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81496) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:11:40.301075 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.301357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.301437 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81497) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:11:40.301638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.301720 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.301923 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.302008 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.302184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.302267 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.302453 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.302544 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.302724 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.302806 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.303017 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.303099 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.303289 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.303371 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.303553 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.303633 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.303825 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.304097 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.326097 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.326784 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.327578 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81496) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:11:40.327863 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.328273 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81497) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:11:40.328558 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.332508 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81496) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:11:40.332791 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.333071 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.333253 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81497) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:11:40.333542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.333826 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.339919 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81496) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:11:40.340205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.340730 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81497) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:11:40.341015 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.345755 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81496) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:11:40.346048 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.346333 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.346619 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.346699 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81497) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:11:40.346920 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.347007 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.347197 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.347281 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.347467 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.347563 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.347737 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.347821 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.348042 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.348319 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.348594 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.363705 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.363985 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.364608 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.364886 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.367843 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81496) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:11:40.368768 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81497) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:11:40.371090 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.371388 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.371978 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.372276 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.375581 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81496) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:11:40.376466 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81497) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:11:40.378703 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81496) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:11:40.379568 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81497) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:11:40.381918 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.382217 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.382500 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.382738 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.382837 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.383065 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.383160 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.383349 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.383429 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.383673 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.383960 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.384239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.399021 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81496) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:11:40.399837 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.400100 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.400362 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.400443 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81497) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:11:40.400679 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.400946 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.401212 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.407448 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.408387 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.408977 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.409929 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.410485 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.410769 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.411482 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.411759 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.413453 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81496) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:11:40.413735 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.414017 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.414294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.414530 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81497) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:11:40.414705 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.415374 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.415669 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.415955 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.416351 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.426298 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81496) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:11:40.426581 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.426882 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.427166 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.427505 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81497) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:11:40.427602 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.427791 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.427876 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.428075 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.428159 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.428365 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.428744 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.429023 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.429315 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.443476 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81496) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:11:40.444750 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81497) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:11:40.446382 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81496) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:11:40.447735 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81497) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:11:40.449285 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.450610 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.450741 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.452143 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.452228 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81496) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:11:40.452428 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.452819 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.453661 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81497) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:11:40.453950 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.454356 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.459147 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.460594 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81496) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:11:40.460775 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.462788 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81497) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:11:40.464038 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.464429 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.464713 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.464990 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.465705 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.466101 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.466387 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.466663 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.472794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81496) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:11:40.473082 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.473360 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.474534 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81497) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:11:40.474837 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.475129 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.480837 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81496) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:11:40.481120 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.481402 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.481677 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.481958 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.482224 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.482493 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.482585 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81497) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 14:11:40.482767 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.482871 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.483066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.483148 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.483341 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.483435 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.483610 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.483693 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.483888 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.483972 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.484161 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.484243 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.484437 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81496) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.484517 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.484705 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.484994 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.485278 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.485547 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.485813 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.486073 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81497) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 14:11:40.518126 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5604dffdd9c0 pid: 81496 (default app) Sep 24 14:11:40.519969 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5604dffdd9c0 pid: 81497 (default app) Sep 24 14:11:40.521318 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf3a3b3-cda1-4b9b-96f9-e7532b74b77a None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:40.521602 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf3a3b3-cda1-4b9b-96f9-e7532b74b77a None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:40.521874 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf3a3b3-cda1-4b9b-96f9-e7532b74b77a None None] PATH_INFO: `/v3` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:40.526575 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1/1] 10.222.0.52 () {50 vars in 791 bytes} [Sat Sep 24 14:11:40 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:11:47.281853 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e99a39-e7dc-4252-8263-7fb5353924ea None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:47.282410 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e99a39-e7dc-4252-8263-7fb5353924ea None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:47.282701 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e99a39-e7dc-4252-8263-7fb5353924ea None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:47.287564 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1/2] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:11:47 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:11:47.293644 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7145f9-aa54-48af-835a-900ea723f83f None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:47.294090 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7145f9-aa54-48af-835a-900ea723f83f None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:47.294224 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7145f9-aa54-48af-835a-900ea723f83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:47.363202 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: INFO dbcounter [None req-fd7145f9-aa54-48af-835a-900ea723f83f None None] Registered counter for database keystone Sep 24 14:11:47.381874 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG oslo_db.sqlalchemy.engines [None req-fd7145f9-aa54-48af-835a-900ea723f83f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81496) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 24 14:11:47.388261 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writer thread running {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 24 14:11:47.582254 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG passlib.handlers.bcrypt [None req-fd7145f9-aa54-48af-835a-900ea723f83f None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81496) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 24 14:11:47.583179 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG passlib.handlers.bcrypt [None req-fd7145f9-aa54-48af-835a-900ea723f83f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81496) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 24 14:11:47.642737 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-fd7145f9-aa54-48af-835a-900ea723f83f None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:11:47.644357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-fd7145f9-aa54-48af-835a-900ea723f83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:11:47.768608 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 2/3] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:11:47 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 476 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 24 14:11:48.213967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e99a39-e7dc-4252-8263-7fb5353924ea None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:11:48.282462 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: INFO dbcounter [None req-f6e99a39-e7dc-4252-8263-7fb5353924ea None None] Registered counter for database keystone Sep 24 14:11:48.303311 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG oslo_db.sqlalchemy.engines [None req-f6e99a39-e7dc-4252-8263-7fb5353924ea None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81497) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 24 14:11:48.309413 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writer thread running {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 24 14:11:48.427646 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e99a39-e7dc-4252-8263-7fb5353924ea None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:11:48.431804 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2709d33-c2d5-4c84-8cb1-219d1833e546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:11:48.432426 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2709d33-c2d5-4c84-8cb1-219d1833e546 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:48.432703 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2709d33-c2d5-4c84-8cb1-219d1833e546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:48.432970 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2709d33-c2d5-4c84-8cb1-219d1833e546 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:48.440968 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2709d33-c2d5-4c84-8cb1-219d1833e546 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:11:48.492825 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b2709d33-c2d5-4c84-8cb1-219d1833e546 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 24 14:11:48.493134 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2709d33-c2d5-4c84-8cb1-219d1833e546 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:11:48.502542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 2/4] 10.222.0.52 () {58 vars in 1324 bytes} [Sat Sep 24 14:11:48 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 24 14:11:48.508444 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7145f9-aa54-48af-835a-900ea723f83f None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:11:48.526206 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7145f9-aa54-48af-835a-900ea723f83f None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:11:48.530935 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a0688-2cf2-417c-ae7e-ac92a5b05739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:11:48.532042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a0688-2cf2-417c-ae7e-ac92a5b05739 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:48.532324 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a0688-2cf2-417c-ae7e-ac92a5b05739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:48.532594 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a0688-2cf2-417c-ae7e-ac92a5b05739 admin admin] PATH_INFO: `/v3/projects/2f8134960517435ea813cbf2d41e25d7` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:48.534164 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a0688-2cf2-417c-ae7e-ac92a5b05739 admin admin] RBAC: Authorizing `identity:get_project(project_id=2f8134960517435ea813cbf2d41e25d7)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:11:48.586786 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: WARNING keystone.common.rbac_enforcer.enforcer [None req-aa1a0688-2cf2-417c-ae7e-ac92a5b05739 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 24 14:11:48.587156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a0688-2cf2-417c-ae7e-ac92a5b05739 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:11:48.591677 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 3/5] 10.222.0.52 () {56 vars in 1207 bytes} [Sat Sep 24 14:11:48 2022] GET /identity/v3/projects/2f8134960517435ea813cbf2d41e25d7 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:11:50.712527 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f035a0-e965-4158-b6fd-2f5d5a7fac5d None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:50.713094 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f035a0-e965-4158-b6fd-2f5d5a7fac5d None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:50.713094 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f035a0-e965-4158-b6fd-2f5d5a7fac5d None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:50.717210 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 3/6] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:11:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:11:50.722858 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bceec0d-d1b6-46e1-a95d-5b8bd6b64f39 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:50.723248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bceec0d-d1b6-46e1-a95d-5b8bd6b64f39 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:50.723410 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bceec0d-d1b6-46e1-a95d-5b8bd6b64f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:50.744112 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-7bceec0d-d1b6-46e1-a95d-5b8bd6b64f39 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:11:50.745626 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-7bceec0d-d1b6-46e1-a95d-5b8bd6b64f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:11:50.753391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 4/7] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:11:50 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 24 14:11:51.021351 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f035a0-e965-4158-b6fd-2f5d5a7fac5d None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:11:51.033541 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f035a0-e965-4158-b6fd-2f5d5a7fac5d None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:11:51.037721 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6f63e-3247-4279-839c-fc881198c360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:11:51.038324 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6f63e-3247-4279-839c-fc881198c360 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:51.038605 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6f63e-3247-4279-839c-fc881198c360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:51.038913 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6f63e-3247-4279-839c-fc881198c360 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:51.046376 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc6f63e-3247-4279-839c-fc881198c360 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:11:51.048184 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc6f63e-3247-4279-839c-fc881198c360 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:11:51.057248 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 4/8] 10.222.0.52 () {58 vars in 1324 bytes} [Sat Sep 24 14:11:51 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 14:11:51.062466 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bceec0d-d1b6-46e1-a95d-5b8bd6b64f39 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:11:51.072068 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bceec0d-d1b6-46e1-a95d-5b8bd6b64f39 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:11:51.076404 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f091740-6c0b-47cc-b2d7-4d6ce966e779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:11:51.077024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f091740-6c0b-47cc-b2d7-4d6ce966e779 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:51.077307 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f091740-6c0b-47cc-b2d7-4d6ce966e779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:51.077582 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f091740-6c0b-47cc-b2d7-4d6ce966e779 admin admin] PATH_INFO: `/v3/users/a6fda065d398415a9b8308457ab0f0dc` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:51.078912 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f091740-6c0b-47cc-b2d7-4d6ce966e779 admin admin] RBAC: Authorizing `identity:get_user(user_id=a6fda065d398415a9b8308457ab0f0dc)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:11:51.080724 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f091740-6c0b-47cc-b2d7-4d6ce966e779 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:11:51.082216 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 5/9] 10.222.0.52 () {56 vars in 1198 bytes} [Sat Sep 24 14:11:51 2022] GET /identity/v3/users/a6fda065d398415a9b8308457ab0f0dc => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:11:53.319964 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30429d1-2baf-48ab-aa2b-49bca52aadd3 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:53.320477 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30429d1-2baf-48ab-aa2b-49bca52aadd3 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:53.320587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30429d1-2baf-48ab-aa2b-49bca52aadd3 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:53.322483 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 5/10] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:11:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:11:53.327727 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56be3b5-f623-45eb-b234-468395ecf6bf None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:53.328141 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56be3b5-f623-45eb-b234-468395ecf6bf None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:53.328391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56be3b5-f623-45eb-b234-468395ecf6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:53.346346 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-d56be3b5-f623-45eb-b234-468395ecf6bf None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:11:53.347768 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-d56be3b5-f623-45eb-b234-468395ecf6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:11:53.358397 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b582e-ced9-4c5a-9a86-fc002ee21f6d None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:53.358922 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b582e-ced9-4c5a-9a86-fc002ee21f6d None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:53.359120 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b582e-ced9-4c5a-9a86-fc002ee21f6d None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:53.360450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 6/11] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:11:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:11:53.365932 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60996d0-ddb7-41c2-8e2c-1202f483a4fe None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:53.366302 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60996d0-ddb7-41c2-8e2c-1202f483a4fe None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:53.366582 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60996d0-ddb7-41c2-8e2c-1202f483a4fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:53.409782 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG passlib.handlers.bcrypt [None req-e60996d0-ddb7-41c2-8e2c-1202f483a4fe None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81497) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 24 14:11:53.410170 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG passlib.handlers.bcrypt [None req-e60996d0-ddb7-41c2-8e2c-1202f483a4fe None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81497) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 24 14:11:53.432057 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 6/12] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:11:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:11:53.459360 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-e60996d0-ddb7-41c2-8e2c-1202f483a4fe None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:11:53.460960 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-e60996d0-ddb7-41c2-8e2c-1202f483a4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:11:53.559425 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 7/13] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:11:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:11:53.705534 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56be3b5-f623-45eb-b234-468395ecf6bf None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:11:53.736023 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:11:53.818103 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56be3b5-f623-45eb-b234-468395ecf6bf None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:11:53.835570 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60996d0-ddb7-41c2-8e2c-1202f483a4fe None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:11:53.854980 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417325b-6c1b-463c-97ad-9cff07eb547b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:11:53.855620 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417325b-6c1b-463c-97ad-9cff07eb547b None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:53.855900 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417325b-6c1b-463c-97ad-9cff07eb547b None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:53.856172 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417325b-6c1b-463c-97ad-9cff07eb547b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:53.867531 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1417325b-6c1b-463c-97ad-9cff07eb547b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:11:53.876346 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-1417325b-6c1b-463c-97ad-9cff07eb547b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:11:53.876346 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:11:53.876346 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:11:53.876346 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:11:53.876346 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:11:53.876346 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:11:53.876346 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:11:53.876346 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:11:53.876346 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:11:53.876346 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:11:53.876346 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:11:53.876346 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:11:53.876346 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:11:53.876346 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:11:53.876346 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:11:53.876346 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:11:53.877227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:11:53.877227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:11:53.877227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:11:53.877227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:11:53.877227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:11:53.877227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:11:53.877227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:11:53.877227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:11:53.877227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:11:53.877227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:11:53.877227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:11:53.877227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:11:53.877227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:11:53.877227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:11:53.877227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:11:53.877227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:11:53.878042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:11:53.878042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:11:53.878042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:11:53.878042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:11:53.878042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:11:53.878042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:11:53.878042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:11:53.878042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:11:53.878042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:11:53.878042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:11:53.878042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:11:53.878042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:11:53.878042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 7/14] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 14:11:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:11:53.883105 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417325b-6c1b-463c-97ad-9cff07eb547b None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:11:53.935433 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60996d0-ddb7-41c2-8e2c-1202f483a4fe None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:11:53.937722 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:11:53.964400 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:11:53.974323 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967557b8-7a3f-4288-a6a4-d26f607a2063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:11:53.975218 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967557b8-7a3f-4288-a6a4-d26f607a2063 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:53.975506 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967557b8-7a3f-4288-a6a4-d26f607a2063 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:53.975785 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967557b8-7a3f-4288-a6a4-d26f607a2063 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:53.983131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417325b-6c1b-463c-97ad-9cff07eb547b None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:11:54.016628 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d38cc3-be7f-4125-bf51-3b7436374af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:11:54.017249 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d38cc3-be7f-4125-bf51-3b7436374af8 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:54.017544 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d38cc3-be7f-4125-bf51-3b7436374af8 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:54.017803 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d38cc3-be7f-4125-bf51-3b7436374af8 None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:54.018476 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d38cc3-be7f-4125-bf51-3b7436374af8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:11:54.018777 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d38cc3-be7f-4125-bf51-3b7436374af8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:11:54.020631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d38cc3-be7f-4125-bf51-3b7436374af8 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:11:54.037579 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 8/15] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:11:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:11:54.043857 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d38cc3-be7f-4125-bf51-3b7436374af8 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:11:54.045588 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967557b8-7a3f-4288-a6a4-d26f607a2063 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:11:54.047149 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967557b8-7a3f-4288-a6a4-d26f607a2063 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:11:54.144894 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d38cc3-be7f-4125-bf51-3b7436374af8 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:11:54.146460 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 8/16] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:11:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 24 14:11:54.151681 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967557b8-7a3f-4288-a6a4-d26f607a2063 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:11:54.155645 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:11:54.179863 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c78ba-190a-4896-b4c4-cb11042e7035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:11:54.180499 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c78ba-190a-4896-b4c4-cb11042e7035 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:54.180787 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c78ba-190a-4896-b4c4-cb11042e7035 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:54.181646 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c78ba-190a-4896-b4c4-cb11042e7035 None admin] PATH_INFO: `/v3/users/a6fda065d398415a9b8308457ab0f0dc` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:54.182785 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c78ba-190a-4896-b4c4-cb11042e7035 None admin] RBAC: Authorizing `identity:get_user(user_id=a6fda065d398415a9b8308457ab0f0dc)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:11:54.184285 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c78ba-190a-4896-b4c4-cb11042e7035 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:11:54.185850 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 9/17] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:11:54 2022] GET /identity/v3/users/a6fda065d398415a9b8308457ab0f0dc => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:11:54.191100 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c78ba-190a-4896-b4c4-cb11042e7035 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:11:54.205711 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:11:54.250709 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967557b8-7a3f-4288-a6a4-d26f607a2063 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:11:54.287121 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ebe53-cd07-4e9c-8e77-7d006918dbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:11:54.287976 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ebe53-cd07-4e9c-8e77-7d006918dbfb None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:54.288253 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ebe53-cd07-4e9c-8e77-7d006918dbfb None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:54.288529 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ebe53-cd07-4e9c-8e77-7d006918dbfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:54.293967 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c78ba-190a-4896-b4c4-cb11042e7035 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:11:54.306804 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730ebe53-cd07-4e9c-8e77-7d006918dbfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:11:54.308285 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730ebe53-cd07-4e9c-8e77-7d006918dbfb None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:11:54.326982 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-730ebe53-cd07-4e9c-8e77-7d006918dbfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:11:54.326982 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:11:54.326982 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:11:54.326982 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:11:54.326982 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:11:54.326982 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:11:54.326982 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:11:54.326982 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:11:54.326982 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:11:54.326982 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:11:54.326982 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:11:54.326982 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:11:54.326982 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:11:54.326982 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:11:54.326982 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:11:54.326982 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:11:54.328267 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:11:54.328267 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:11:54.328267 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:11:54.328267 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:11:54.328267 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:11:54.328267 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:11:54.328267 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:11:54.328267 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:11:54.328267 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:11:54.328267 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:11:54.328267 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:11:54.328267 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:11:54.328267 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:11:54.328267 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:11:54.328267 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:11:54.328267 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:11:54.328267 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:11:54.329432 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:11:54.329432 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:11:54.329432 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:11:54.329432 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 9/18] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:11:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:11:54.329655 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a04008-2cb4-49fe-a988-8099b067ff4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:11:54.329655 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a04008-2cb4-49fe-a988-8099b067ff4f None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:54.329655 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a04008-2cb4-49fe-a988-8099b067ff4f None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:54.329655 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a04008-2cb4-49fe-a988-8099b067ff4f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:54.330623 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a04008-2cb4-49fe-a988-8099b067ff4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:11:54.332115 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a04008-2cb4-49fe-a988-8099b067ff4f None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:11:54.333873 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ebe53-cd07-4e9c-8e77-7d006918dbfb None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:11:54.333983 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 10/19] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:11:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:11:54.340259 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a04008-2cb4-49fe-a988-8099b067ff4f None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:11:54.390092 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:11:54.421232 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:11:54.435221 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ebe53-cd07-4e9c-8e77-7d006918dbfb None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:11:54.440086 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a04008-2cb4-49fe-a988-8099b067ff4f None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:11:54.468647 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec3e83e-6821-4b18-82f8-6bc28f6deefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:11:54.469250 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3e83e-6821-4b18-82f8-6bc28f6deefe None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:54.469528 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3e83e-6821-4b18-82f8-6bc28f6deefe None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:54.469799 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3e83e-6821-4b18-82f8-6bc28f6deefe None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:54.470463 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec3e83e-6821-4b18-82f8-6bc28f6deefe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:11:54.470765 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec3e83e-6821-4b18-82f8-6bc28f6deefe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:11:54.472190 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec3e83e-6821-4b18-82f8-6bc28f6deefe None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:11:54.473869 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ce955-a18b-41e1-88de-eaf5c17b69f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:11:54.474480 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ce955-a18b-41e1-88de-eaf5c17b69f2 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:54.474760 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ce955-a18b-41e1-88de-eaf5c17b69f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:54.475063 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ce955-a18b-41e1-88de-eaf5c17b69f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:54.475787 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16ce955-a18b-41e1-88de-eaf5c17b69f2 None admin] RBAC: Adding query filter params (role.id=d00b78c32dbf4cc7925a55bb621d8fcf, scope.domain.id=default, user.id=a6fda065d398415a9b8308457ab0f0dc) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:11:54.476103 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16ce955-a18b-41e1-88de-eaf5c17b69f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:11:54.477535 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16ce955-a18b-41e1-88de-eaf5c17b69f2 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:11:54.484771 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 11/20] 10.222.0.52 () {56 vars in 1322 bytes} [Sat Sep 24 14:11:54 2022] GET /identity/v3/role_assignments?user.id=a6fda065d398415a9b8308457ab0f0dc&scope.domain.id=default&role.id=d00b78c32dbf4cc7925a55bb621d8fcf => generated 233 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:11:54.492338 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 10/21] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:11:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:11:57.232927 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b667e22a-b11c-4aae-9d1c-267825659b4a None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:57.233432 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b667e22a-b11c-4aae-9d1c-267825659b4a None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:57.233506 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b667e22a-b11c-4aae-9d1c-267825659b4a None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:57.234798 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 12/22] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:11:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:11:57.240298 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc2926-bcc0-4b08-a5bd-7ccda171be9c None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:57.241146 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc2926-bcc0-4b08-a5bd-7ccda171be9c None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:57.241640 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc2926-bcc0-4b08-a5bd-7ccda171be9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:57.271839 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-90cc2926-bcc0-4b08-a5bd-7ccda171be9c None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:11:57.273457 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-90cc2926-bcc0-4b08-a5bd-7ccda171be9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:11:57.278245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583195e9-1e60-4755-8f7c-4f9407e3063b None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:57.278993 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583195e9-1e60-4755-8f7c-4f9407e3063b None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:57.279722 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583195e9-1e60-4755-8f7c-4f9407e3063b None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:57.281367 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 13/23] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:11:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:11:57.288187 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c118af72-95b7-4176-a798-b4f746394d49 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:57.288793 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c118af72-95b7-4176-a798-b4f746394d49 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:57.289293 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c118af72-95b7-4176-a798-b4f746394d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:57.316878 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-c118af72-95b7-4176-a798-b4f746394d49 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:11:57.319224 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-c118af72-95b7-4176-a798-b4f746394d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:11:57.420386 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 11/24] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:11:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:11:57.445288 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 14/25] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:11:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:11:57.550782 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b13ac4-0bfc-4c33-99b1-bd586108be3b None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:57.551413 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b13ac4-0bfc-4c33-99b1-bd586108be3b None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:57.551891 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b13ac4-0bfc-4c33-99b1-bd586108be3b None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:57.553443 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 12/26] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:11:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:11:57.559321 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01078a5a-8715-44c3-9406-8d37742d0c55 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:57.559846 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01078a5a-8715-44c3-9406-8d37742d0c55 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:57.560320 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01078a5a-8715-44c3-9406-8d37742d0c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:57.587848 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-01078a5a-8715-44c3-9406-8d37742d0c55 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:11:57.589352 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-01078a5a-8715-44c3-9406-8d37742d0c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:11:57.601327 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac42c15-1388-4ba4-9b3f-f20fea44db05 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:57.601897 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac42c15-1388-4ba4-9b3f-f20fea44db05 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:57.602394 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac42c15-1388-4ba4-9b3f-f20fea44db05 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:57.603991 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 13/27] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:11:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:11:57.609187 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb8743-4960-42dd-a42a-c1b3a77ce934 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:57.609704 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb8743-4960-42dd-a42a-c1b3a77ce934 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:57.610177 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb8743-4960-42dd-a42a-c1b3a77ce934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:57.640080 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-34bb8743-4960-42dd-a42a-c1b3a77ce934 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:11:57.641590 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-34bb8743-4960-42dd-a42a-c1b3a77ce934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:11:57.717965 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 15/28] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:11:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:11:57.720279 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:11:57.747121 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 14/29] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:11:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:11:57.873646 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01078a5a-8715-44c3-9406-8d37742d0c55 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:11:57.877681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:11:57.995782 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01078a5a-8715-44c3-9406-8d37742d0c55 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:11:58.030792 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb8743-4960-42dd-a42a-c1b3a77ce934 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:11:58.050665 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d7cb17-f02d-49cf-8e2e-460e349063fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:11:58.051643 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7cb17-f02d-49cf-8e2e-460e349063fb None admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:58.052137 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7cb17-f02d-49cf-8e2e-460e349063fb None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:58.053765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7cb17-f02d-49cf-8e2e-460e349063fb None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:58.054649 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d7cb17-f02d-49cf-8e2e-460e349063fb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:11:58.057142 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d7cb17-f02d-49cf-8e2e-460e349063fb None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:11:58.201005 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb8743-4960-42dd-a42a-c1b3a77ce934 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:11:58.218648 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 16/30] 10.222.0.52 () {60 vars in 1131 bytes} [Sat Sep 24 14:11:57 2022] POST /identity/v3/roles => generated 219 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:11:58.221797 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d7cb17-f02d-49cf-8e2e-460e349063fb None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:11:58.287155 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6b011d-a364-45ee-890c-518fbe4d7a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:11:58.287868 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6b011d-a364-45ee-890c-518fbe4d7a4e None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:58.288112 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6b011d-a364-45ee-890c-518fbe4d7a4e None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:58.288383 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6b011d-a364-45ee-890c-518fbe4d7a4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:58.316696 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6b011d-a364-45ee-890c-518fbe4d7a4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:11:58.318264 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6b011d-a364-45ee-890c-518fbe4d7a4e None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:11:58.348692 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-5d6b011d-a364-45ee-890c-518fbe4d7a4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:11:58.348692 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:11:58.348692 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:11:58.348692 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:11:58.348692 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:11:58.348692 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:11:58.348692 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:11:58.348692 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:11:58.348692 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:11:58.348692 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:11:58.348692 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:11:58.348692 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:11:58.348692 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:11:58.348692 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:11:58.348692 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:11:58.348692 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:11:58.349611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:11:58.349611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:11:58.349611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:11:58.349611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:11:58.349611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:11:58.349611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:11:58.349611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:11:58.349611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:11:58.349611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:11:58.349611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:11:58.349611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:11:58.349611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:11:58.349611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:11:58.349611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:11:58.349611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:11:58.349611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:11:58.349611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:11:58.350406 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:11:58.350406 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:11:58.350406 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:11:58.350406 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 15/31] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:11:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:11:58.351430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6b011d-a364-45ee-890c-518fbe4d7a4e None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:11:58.363456 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:11:58.400059 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:11:58.432597 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d7cb17-f02d-49cf-8e2e-460e349063fb None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:11:58.472985 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5963a0-4668-4bdc-a20b-cb63bc5f8173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:11:58.473865 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5963a0-4668-4bdc-a20b-cb63bc5f8173 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:58.474209 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5963a0-4668-4bdc-a20b-cb63bc5f8173 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:58.474748 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5963a0-4668-4bdc-a20b-cb63bc5f8173 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:58.481320 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6b011d-a364-45ee-890c-518fbe4d7a4e None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:11:58.492912 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5963a0-4668-4bdc-a20b-cb63bc5f8173 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:11:58.500986 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-7a5963a0-4668-4bdc-a20b-cb63bc5f8173 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:11:58.500986 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:11:58.500986 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:11:58.500986 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:11:58.500986 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:11:58.500986 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:11:58.500986 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:11:58.500986 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:11:58.500986 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:11:58.500986 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:11:58.500986 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:11:58.500986 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:11:58.500986 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:11:58.500986 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:11:58.500986 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:11:58.500986 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:11:58.502027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:11:58.502027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:11:58.502027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:11:58.502027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:11:58.502027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:11:58.502027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:11:58.502027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:11:58.502027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:11:58.502027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:11:58.502027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:11:58.502027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:11:58.502027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:11:58.502027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:11:58.502027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:11:58.502027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:11:58.502027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:11:58.502955 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:11:58.502955 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:11:58.502955 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:11:58.502955 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:11:58.502955 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:11:58.502955 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:11:58.502955 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:11:58.502955 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:11:58.502955 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:11:58.502955 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:11:58.502955 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:11:58.502955 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:11:58.504296 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 17/32] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 14:11:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:11:58.509812 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5963a0-4668-4bdc-a20b-cb63bc5f8173 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:11:58.530478 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7285bf3-e11f-46c2-b3f9-1f90c4535479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:11:58.531208 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7285bf3-e11f-46c2-b3f9-1f90c4535479 None admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:58.531490 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7285bf3-e11f-46c2-b3f9-1f90c4535479 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:58.531762 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7285bf3-e11f-46c2-b3f9-1f90c4535479 None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:58.532438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7285bf3-e11f-46c2-b3f9-1f90c4535479 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:11:58.533853 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7285bf3-e11f-46c2-b3f9-1f90c4535479 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:11:58.612155 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5963a0-4668-4bdc-a20b-cb63bc5f8173 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:11:58.635849 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:11:58.647909 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e78796-4e49-450b-859e-4689bfe9198f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:11:58.648519 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e78796-4e49-450b-859e-4689bfe9198f None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:58.648803 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e78796-4e49-450b-859e-4689bfe9198f None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:58.649078 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e78796-4e49-450b-859e-4689bfe9198f None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:58.650241 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e78796-4e49-450b-859e-4689bfe9198f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:11:58.650556 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e78796-4e49-450b-859e-4689bfe9198f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:11:58.652091 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e78796-4e49-450b-859e-4689bfe9198f None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:11:58.656656 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 16/33] 10.222.0.52 () {60 vars in 1131 bytes} [Sat Sep 24 14:11:58 2022] POST /identity/v3/roles => generated 225 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:11:58.657777 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab5cec-b65f-4f37-ba31-901a10794217 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:58.658070 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab5cec-b65f-4f37-ba31-901a10794217 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:58.658353 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab5cec-b65f-4f37-ba31-901a10794217 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:58.659605 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 17/34] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:11:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:11:58.664331 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab5cec-b65f-4f37-ba31-901a10794217 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:11:58.679223 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 18/35] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:11:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:11:58.680485 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf858e0f-286e-4909-bd7f-b3d5e247b2e6 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:58.680779 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf858e0f-286e-4909-bd7f-b3d5e247b2e6 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:58.681108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf858e0f-286e-4909-bd7f-b3d5e247b2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:58.703086 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-cf858e0f-286e-4909-bd7f-b3d5e247b2e6 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:11:58.704768 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-cf858e0f-286e-4909-bd7f-b3d5e247b2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:11:58.794971 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 19/36] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:11:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:11:58.794971 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf858e0f-286e-4909-bd7f-b3d5e247b2e6 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:11:58.825509 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:11:58.854968 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab5cec-b65f-4f37-ba31-901a10794217 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:11:58.896355 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f7df9-9cab-4198-877b-e219a6bd51eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:11:58.896984 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f7df9-9cab-4198-877b-e219a6bd51eb None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:58.897268 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f7df9-9cab-4198-877b-e219a6bd51eb None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:58.897543 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f7df9-9cab-4198-877b-e219a6bd51eb None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:58.898223 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185f7df9-9cab-4198-877b-e219a6bd51eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:11:58.899267 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185f7df9-9cab-4198-877b-e219a6bd51eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:11:58.900124 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185f7df9-9cab-4198-877b-e219a6bd51eb None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:11:58.912649 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 18/37] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:11:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:11:58.914025 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed3ba6b-cb1f-41b7-acb0-51ec1355b2ee None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:58.914318 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed3ba6b-cb1f-41b7-acb0-51ec1355b2ee None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:58.914599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed3ba6b-cb1f-41b7-acb0-51ec1355b2ee None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:58.916515 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 19/38] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:11:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:11:58.921257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed3ba6b-cb1f-41b7-acb0-51ec1355b2ee None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:11:58.937590 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf858e0f-286e-4909-bd7f-b3d5e247b2e6 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:11:58.955026 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:11:58.979423 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d358e15-36b0-4f05-a5bc-1a4ebba4b2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:11:58.981777 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d358e15-36b0-4f05-a5bc-1a4ebba4b2f8 None admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:58.981777 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d358e15-36b0-4f05-a5bc-1a4ebba4b2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:58.981777 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d358e15-36b0-4f05-a5bc-1a4ebba4b2f8 None admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:58.981777 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d358e15-36b0-4f05-a5bc-1a4ebba4b2f8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:11:58.982689 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d358e15-36b0-4f05-a5bc-1a4ebba4b2f8 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:11:59.021657 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 20/39] 10.222.0.52 () {60 vars in 1141 bytes} [Sat Sep 24 14:11:58 2022] POST /identity/v3/projects => generated 301 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:11:59.021657 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c5578-5f14-418d-801f-61cfd592db05 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:59.021657 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c5578-5f14-418d-801f-61cfd592db05 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:59.021657 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c5578-5f14-418d-801f-61cfd592db05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:59.040177 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-4b9c5578-5f14-418d-801f-61cfd592db05 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:11:59.041710 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-4b9c5578-5f14-418d-801f-61cfd592db05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:11:59.070346 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed3ba6b-cb1f-41b7-acb0-51ec1355b2ee None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:11:59.117323 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d64d6e0-55ab-4ae0-b199-9a39ff809470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:11:59.117981 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d64d6e0-55ab-4ae0-b199-9a39ff809470 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:59.118265 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d64d6e0-55ab-4ae0-b199-9a39ff809470 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:59.119122 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d64d6e0-55ab-4ae0-b199-9a39ff809470 None admin] PATH_INFO: `/v3/users/a6fda065d398415a9b8308457ab0f0dc` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:59.120801 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d64d6e0-55ab-4ae0-b199-9a39ff809470 None admin] RBAC: Authorizing `identity:get_user(user_id=a6fda065d398415a9b8308457ab0f0dc)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:11:59.122326 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d64d6e0-55ab-4ae0-b199-9a39ff809470 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:11:59.124027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 20/40] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:11:58 2022] GET /identity/v3/users/a6fda065d398415a9b8308457ab0f0dc => generated 247 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:11:59.127041 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d64d6e0-55ab-4ae0-b199-9a39ff809470 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:11:59.142863 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 21/41] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:11:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:11:59.143592 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c5578-5f14-418d-801f-61cfd592db05 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:11:59.151839 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:11:59.265085 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d64d6e0-55ab-4ae0-b199-9a39ff809470 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:11:59.303207 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fa2958-16e7-4d65-8260-8b1409ebb151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:11:59.303882 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa2958-16e7-4d65-8260-8b1409ebb151 None admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:59.304195 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa2958-16e7-4d65-8260-8b1409ebb151 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:59.304373 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa2958-16e7-4d65-8260-8b1409ebb151 None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:59.305057 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fa2958-16e7-4d65-8260-8b1409ebb151 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:11:59.307025 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fa2958-16e7-4d65-8260-8b1409ebb151 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:11:59.322833 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 21/42] 10.222.0.52 () {60 vars in 1131 bytes} [Sat Sep 24 14:11:59 2022] POST /identity/v3/roles => generated 223 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:11:59.372364 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:11:59.379812 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c5578-5f14-418d-801f-61cfd592db05 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:11:59.421810 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1fb080-3b4c-4dc4-bd65-3b1ee20d3601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:11:59.422656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1fb080-3b4c-4dc4-bd65-3b1ee20d3601 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:59.422982 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1fb080-3b4c-4dc4-bd65-3b1ee20d3601 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:59.423262 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1fb080-3b4c-4dc4-bd65-3b1ee20d3601 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:59.424465 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1fb080-3b4c-4dc4-bd65-3b1ee20d3601 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:11:59.424670 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fa2958-16e7-4d65-8260-8b1409ebb151 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:11:59.425966 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1fb080-3b4c-4dc4-bd65-3b1ee20d3601 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:11:59.427703 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 22/43] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:11:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:11:59.435290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1fb080-3b4c-4dc4-bd65-3b1ee20d3601 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:11:59.524579 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:11:59.552807 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1fb080-3b4c-4dc4-bd65-3b1ee20d3601 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:11:59.555445 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fa2958-16e7-4d65-8260-8b1409ebb151 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:11:59.586645 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2edc89-34e6-4798-8c8c-11eb858eeb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:11:59.587806 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2edc89-34e6-4798-8c8c-11eb858eeb29 None admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:59.588084 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2edc89-34e6-4798-8c8c-11eb858eeb29 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:59.588357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2edc89-34e6-4798-8c8c-11eb858eeb29 None admin] PATH_INFO: `/v3/domains/default/users/a6fda065d398415a9b8308457ab0f0dc/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:59.589645 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1da100-096e-4618-8604-45d85606a3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:11:59.590247 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2edc89-34e6-4798-8c8c-11eb858eeb29 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=a6fda065d398415a9b8308457ab0f0dc, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:11:59.590356 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1da100-096e-4618-8604-45d85606a3ff None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:59.590576 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1da100-096e-4618-8604-45d85606a3ff None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:59.590874 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1da100-096e-4618-8604-45d85606a3ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:59.591727 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2edc89-34e6-4798-8c8c-11eb858eeb29 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:11:59.592073 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1da100-096e-4618-8604-45d85606a3ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:11:59.593497 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1da100-096e-4618-8604-45d85606a3ff None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:11:59.595106 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 22/44] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:11:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:11:59.600883 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1da100-096e-4618-8604-45d85606a3ff None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:11:59.605392 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 23/45] 10.222.0.52 () {58 vars in 1361 bytes} [Sat Sep 24 14:11:59 2022] PUT /identity/v3/domains/default/users/a6fda065d398415a9b8308457ab0f0dc/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:11:59.706377 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1da100-096e-4618-8604-45d85606a3ff None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:11:59.742794 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f5e52-b938-4f2b-8d2d-4438217903f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:11:59.743448 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f5e52-b938-4f2b-8d2d-4438217903f1 None admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:11:59.743734 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f5e52-b938-4f2b-8d2d-4438217903f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:11:59.744011 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f5e52-b938-4f2b-8d2d-4438217903f1 None admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:11:59.744679 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48f5e52-b938-4f2b-8d2d-4438217903f1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:11:59.746170 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48f5e52-b938-4f2b-8d2d-4438217903f1 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:11:59.771872 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:11:59.781304 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 23/46] 10.222.0.52 () {60 vars in 1141 bytes} [Sat Sep 24 14:11:59 2022] POST /identity/v3/projects => generated 312 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:12:01.762395 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b223db6-19dc-42ad-98a0-67b5782a003e None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:01.762973 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b223db6-19dc-42ad-98a0-67b5782a003e None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:01.762973 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b223db6-19dc-42ad-98a0-67b5782a003e None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:01.764336 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 24/47] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:01.769523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeea8c0-ce29-47d8-90d1-e11880008779 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:01.769901 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeea8c0-ce29-47d8-90d1-e11880008779 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:01.770083 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeea8c0-ce29-47d8-90d1-e11880008779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:01.788558 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-1aeea8c0-ce29-47d8-90d1-e11880008779 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:01.790064 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-1aeea8c0-ce29-47d8-90d1-e11880008779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:01.865306 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 24/48] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:01.936102 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbf2dc-2a59-4932-bed4-d8698e43ca5a None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:01.936387 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbf2dc-2a59-4932-bed4-d8698e43ca5a None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:01.936666 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbf2dc-2a59-4932-bed4-d8698e43ca5a None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:01.938176 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 25/49] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:01.943139 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e204dd5-d49a-41b8-a627-d9688202d696 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:01.943434 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e204dd5-d49a-41b8-a627-d9688202d696 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:01.943712 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e204dd5-d49a-41b8-a627-d9688202d696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:01.962418 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-4e204dd5-d49a-41b8-a627-d9688202d696 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:01.963644 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-4e204dd5-d49a-41b8-a627-d9688202d696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:02.030399 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 25/50] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:12:02.136484 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edbf2dc-2a59-4932-bed4-d8698e43ca5a None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:02.166853 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:02.247398 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edbf2dc-2a59-4932-bed4-d8698e43ca5a None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:02.282972 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae5aa00-c62d-4c5d-b97e-beb1b4116744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:02.283528 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5aa00-c62d-4c5d-b97e-beb1b4116744 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:02.283806 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5aa00-c62d-4c5d-b97e-beb1b4116744 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:02.284079 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5aa00-c62d-4c5d-b97e-beb1b4116744 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:02.295035 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae5aa00-c62d-4c5d-b97e-beb1b4116744 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:02.299807 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-3ae5aa00-c62d-4c5d-b97e-beb1b4116744 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:12:02.299807 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:02.299807 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:02.299807 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:02.299807 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:02.299807 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:02.299807 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:02.299807 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:02.299807 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:02.299807 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:02.299807 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:02.299807 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:02.299807 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:02.299807 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:02.299807 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:02.299807 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:02.300636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:02.300636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:02.300636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:02.300636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:02.300636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:02.300636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:02.300636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:02.300636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:02.300636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:02.300636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:02.300636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:02.300636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:02.300636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:02.300636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:02.300636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:02.300636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:02.301357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:02.301357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:02.301357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:02.301357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:02.301357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:02.301357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:02.301357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:02.301357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:02.301357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:02.301357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:02.301357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:12:02.301357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:12:02.301357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 26/51] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 14:12:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:12:02.307416 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e204dd5-d49a-41b8-a627-d9688202d696 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:02.308623 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae5aa00-c62d-4c5d-b97e-beb1b4116744 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:02.365077 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:02.398395 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:02.411765 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e204dd5-d49a-41b8-a627-d9688202d696 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:02.414792 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae5aa00-c62d-4c5d-b97e-beb1b4116744 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:02.446432 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae5ab35-9797-4106-88e2-e4b1b2c23e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:02.447085 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5ab35-9797-4106-88e2-e4b1b2c23e21 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:02.447375 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5ab35-9797-4106-88e2-e4b1b2c23e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:02.447637 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5ab35-9797-4106-88e2-e4b1b2c23e21 None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:02.448295 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae5ab35-9797-4106-88e2-e4b1b2c23e21 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:02.448605 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae5ab35-9797-4106-88e2-e4b1b2c23e21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:02.449991 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae5ab35-9797-4106-88e2-e4b1b2c23e21 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:02.450620 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd10f5-285f-4499-a6a9-0be0879c5504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:02.451295 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd10f5-285f-4499-a6a9-0be0879c5504 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:02.451582 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd10f5-285f-4499-a6a9-0be0879c5504 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:02.451857 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd10f5-285f-4499-a6a9-0be0879c5504 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:02.453022 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdd10f5-285f-4499-a6a9-0be0879c5504 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:02.454532 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdd10f5-285f-4499-a6a9-0be0879c5504 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:02.456127 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 27/52] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:12:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:02.459834 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 26/53] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:12:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:02.461992 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd10f5-285f-4499-a6a9-0be0879c5504 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:02.465213 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae5ab35-9797-4106-88e2-e4b1b2c23e21 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:02.568788 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd10f5-285f-4499-a6a9-0be0879c5504 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:02.606227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820865c-fa8a-4db1-b712-017119ae6530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:02.606921 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820865c-fa8a-4db1-b712-017119ae6530 None admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:02.607208 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820865c-fa8a-4db1-b712-017119ae6530 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:02.607480 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820865c-fa8a-4db1-b712-017119ae6530 None admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:02.608158 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9820865c-fa8a-4db1-b712-017119ae6530 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:02.609648 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9820865c-fa8a-4db1-b712-017119ae6530 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:02.637732 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 28/54] 10.222.0.52 () {60 vars in 1140 bytes} [Sat Sep 24 14:12:02 2022] POST /identity/v3/projects => generated 298 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:12:02.672414 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae5ab35-9797-4106-88e2-e4b1b2c23e21 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:02.695415 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:02.711397 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d562e6dd-b277-464c-9f4d-686f1724f701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:02.712052 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562e6dd-b277-464c-9f4d-686f1724f701 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:02.712334 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562e6dd-b277-464c-9f4d-686f1724f701 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:02.712610 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562e6dd-b277-464c-9f4d-686f1724f701 None admin] PATH_INFO: `/v3/users/a6fda065d398415a9b8308457ab0f0dc` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:02.713794 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d562e6dd-b277-464c-9f4d-686f1724f701 None admin] RBAC: Authorizing `identity:get_user(user_id=a6fda065d398415a9b8308457ab0f0dc)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:02.715323 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d562e6dd-b277-464c-9f4d-686f1724f701 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:02.716971 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 27/55] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:12:02 2022] GET /identity/v3/users/a6fda065d398415a9b8308457ab0f0dc => generated 247 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:02.723260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820865c-fa8a-4db1-b712-017119ae6530 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:02.727406 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:02.837621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820865c-fa8a-4db1-b712-017119ae6530 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:02.872902 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8318b0c4-d054-4dee-abc0-75c22aebac1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:02.873544 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318b0c4-d054-4dee-abc0-75c22aebac1c None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:02.873828 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318b0c4-d054-4dee-abc0-75c22aebac1c None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:02.874104 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318b0c4-d054-4dee-abc0-75c22aebac1c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:02.875238 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8318b0c4-d054-4dee-abc0-75c22aebac1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:02.876685 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8318b0c4-d054-4dee-abc0-75c22aebac1c None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:02.878244 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 29/56] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:12:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:02.884755 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d562e6dd-b277-464c-9f4d-686f1724f701 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:02.981346 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d562e6dd-b277-464c-9f4d-686f1724f701 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:03.014979 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ead4c3-8c67-4963-9cc8-7437104d4024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:03.015589 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ead4c3-8c67-4963-9cc8-7437104d4024 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:03.015869 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ead4c3-8c67-4963-9cc8-7437104d4024 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:03.016145 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ead4c3-8c67-4963-9cc8-7437104d4024 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:03.016870 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ead4c3-8c67-4963-9cc8-7437104d4024 None admin] RBAC: Adding query filter params (role.id=d00b78c32dbf4cc7925a55bb621d8fcf, scope.domain.id=default, user.id=a6fda065d398415a9b8308457ab0f0dc) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:03.017179 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ead4c3-8c67-4963-9cc8-7437104d4024 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:03.018616 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ead4c3-8c67-4963-9cc8-7437104d4024 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:03.027580 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 28/57] 10.222.0.52 () {56 vars in 1322 bytes} [Sat Sep 24 14:12:02 2022] GET /identity/v3/role_assignments?user.id=a6fda065d398415a9b8308457ab0f0dc&scope.domain.id=default&role.id=d00b78c32dbf4cc7925a55bb621d8fcf => generated 530 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:04.780807 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1260a9f-cc25-4653-8fe1-f23039e968e8 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:04.781341 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1260a9f-cc25-4653-8fe1-f23039e968e8 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:04.781341 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1260a9f-cc25-4653-8fe1-f23039e968e8 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:04.782688 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 30/58] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:04.787912 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44cd38-a2bd-4dc0-90bc-a795cc098775 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:04.788334 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44cd38-a2bd-4dc0-90bc-a795cc098775 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:04.788469 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44cd38-a2bd-4dc0-90bc-a795cc098775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:04.805907 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-bf44cd38-a2bd-4dc0-90bc-a795cc098775 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:04.807142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-bf44cd38-a2bd-4dc0-90bc-a795cc098775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:04.853275 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:04.872652 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 29/59] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:05.140219 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1260a9f-cc25-4653-8fe1-f23039e968e8 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:05.209290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:05.239252 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1260a9f-cc25-4653-8fe1-f23039e968e8 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:05.271966 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161835ed-328c-4f9a-a5bc-5cb0766493ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:05.272584 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161835ed-328c-4f9a-a5bc-5cb0766493ca None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:05.272864 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161835ed-328c-4f9a-a5bc-5cb0766493ca None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:05.273135 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161835ed-328c-4f9a-a5bc-5cb0766493ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:05.274311 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161835ed-328c-4f9a-a5bc-5cb0766493ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:05.275823 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161835ed-328c-4f9a-a5bc-5cb0766493ca None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:05.277367 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 31/60] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:12:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:05.283097 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf44cd38-a2bd-4dc0-90bc-a795cc098775 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:05.376777 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf44cd38-a2bd-4dc0-90bc-a795cc098775 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:05.409347 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6748d8e4-b106-4aba-ae21-a5ff3b9f6bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:05.409879 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748d8e4-b106-4aba-ae21-a5ff3b9f6bab None admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:05.410158 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748d8e4-b106-4aba-ae21-a5ff3b9f6bab None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:05.410430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748d8e4-b106-4aba-ae21-a5ff3b9f6bab None admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:05.411108 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6748d8e4-b106-4aba-ae21-a5ff3b9f6bab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:05.412548 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6748d8e4-b106-4aba-ae21-a5ff3b9f6bab None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:05.438633 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 30/61] 10.222.0.52 () {60 vars in 1132 bytes} [Sat Sep 24 14:12:05 2022] POST /identity/v3/users => generated 275 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:12:08.704024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d8f94-1c47-4c55-8474-356ba2ded5e4 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:08.704530 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d8f94-1c47-4c55-8474-356ba2ded5e4 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:08.704601 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d8f94-1c47-4c55-8474-356ba2ded5e4 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:08.706172 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 32/62] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:08.715715 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4b31b-61b0-4a88-9c8c-8cbeaf516250 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:08.716535 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4b31b-61b0-4a88-9c8c-8cbeaf516250 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:08.717001 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4b31b-61b0-4a88-9c8c-8cbeaf516250 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:08.718557 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 31/63] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:08.721681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6579f-a3e3-4161-9731-1dbd9d1ea33b None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:08.722207 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6579f-a3e3-4161-9731-1dbd9d1ea33b None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:08.722672 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6579f-a3e3-4161-9731-1dbd9d1ea33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:08.731025 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78168427-0462-4697-9b50-ab0685595f78 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:08.732980 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78168427-0462-4697-9b50-ab0685595f78 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:08.734256 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78168427-0462-4697-9b50-ab0685595f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:08.748723 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-a2a6579f-a3e3-4161-9731-1dbd9d1ea33b None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:08.751608 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-a2a6579f-a3e3-4161-9731-1dbd9d1ea33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:08.771668 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-78168427-0462-4697-9b50-ab0685595f78 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:08.774322 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-78168427-0462-4697-9b50-ab0685595f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:08.832871 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:08.879297 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 33/64] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:08.889271 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 32/65] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:09.065776 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59369e05-01fe-4ebd-88bb-bc1ef0f6637e None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:09.066317 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59369e05-01fe-4ebd-88bb-bc1ef0f6637e None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:09.066851 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59369e05-01fe-4ebd-88bb-bc1ef0f6637e None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:09.068370 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 34/66] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:09.073833 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e27fde-aaad-4ab6-9246-0a5c58d15773 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:09.074302 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e27fde-aaad-4ab6-9246-0a5c58d15773 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:09.074741 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e27fde-aaad-4ab6-9246-0a5c58d15773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:09.104472 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-b4e27fde-aaad-4ab6-9246-0a5c58d15773 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:09.105915 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-b4e27fde-aaad-4ab6-9246-0a5c58d15773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:09.178909 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59369e05-01fe-4ebd-88bb-bc1ef0f6637e None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:09.192709 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 33/67] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:12:09.314264 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e27fde-aaad-4ab6-9246-0a5c58d15773 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:09.324594 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:09.369122 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59369e05-01fe-4ebd-88bb-bc1ef0f6637e None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:09.408723 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65efed-f30b-48bd-8b68-053d18cbbd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:09.409590 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65efed-f30b-48bd-8b68-053d18cbbd10 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:09.410010 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65efed-f30b-48bd-8b68-053d18cbbd10 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:09.410421 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65efed-f30b-48bd-8b68-053d18cbbd10 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:09.421124 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65efed-f30b-48bd-8b68-053d18cbbd10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:09.426173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-bd65efed-f30b-48bd-8b68-053d18cbbd10 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:12:09.426173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:09.426173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:09.426173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:09.426173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:09.426173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:09.426173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:09.426173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:09.426173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:09.426173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:09.426173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:09.426173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:09.426173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:09.426173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:09.426173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:09.426173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:09.427035 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:09.427035 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:09.427035 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:09.427035 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:09.427035 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:09.427035 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:09.427035 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:09.427035 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:09.427035 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:09.427035 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:09.427035 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:09.427035 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:09.427035 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:09.427035 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:09.427035 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:09.427035 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:09.427784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:09.427784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:09.427784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:09.427784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:09.427784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:09.427784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:09.427784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:09.427784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:09.427784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:09.427784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:09.427784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:12:09.427784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:12:09.429995 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 35/68] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:12:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:12:09.431895 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c138110-d628-4a7c-ae8c-41e759a8d7ad None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:09.432386 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c138110-d628-4a7c-ae8c-41e759a8d7ad None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:09.432852 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c138110-d628-4a7c-ae8c-41e759a8d7ad None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:09.434298 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 36/69] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:09.435218 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e27fde-aaad-4ab6-9246-0a5c58d15773 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:09.436460 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c138110-d628-4a7c-ae8c-41e759a8d7ad None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:09.438556 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:09.475338 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf60983-a785-40e9-b5c5-64f8ce300d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:09.476004 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf60983-a785-40e9-b5c5-64f8ce300d3a None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:09.476284 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf60983-a785-40e9-b5c5-64f8ce300d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:09.476558 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf60983-a785-40e9-b5c5-64f8ce300d3a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:09.489045 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf60983-a785-40e9-b5c5-64f8ce300d3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:09.496689 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-2cf60983-a785-40e9-b5c5-64f8ce300d3a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:12:09.496689 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:09.496689 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:09.496689 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:09.496689 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:09.496689 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:09.496689 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:09.496689 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:09.496689 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:09.496689 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:09.496689 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:09.496689 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:09.496689 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:09.496689 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:09.496689 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:09.496689 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:09.497573 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:09.497573 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:09.497573 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:09.497573 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:09.497573 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:09.497573 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:09.497573 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:09.497573 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:09.497573 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:09.497573 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:09.497573 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:09.497573 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:09.497573 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:09.497573 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:09.497573 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:09.497573 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:09.503420 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:09.503420 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:09.503420 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:09.503420 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:09.503420 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:09.503420 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:09.503420 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:09.503420 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:09.503420 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:09.503420 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:09.503420 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:12:09.503420 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:12:09.503420 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 34/70] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 14:12:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:12:09.503420 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2059863-79e0-4987-852d-40d1564be833 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:09.504126 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2059863-79e0-4987-852d-40d1564be833 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:09.504126 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2059863-79e0-4987-852d-40d1564be833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:09.519668 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-c2059863-79e0-4987-852d-40d1564be833 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:09.520844 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-c2059863-79e0-4987-852d-40d1564be833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:09.558544 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c138110-d628-4a7c-ae8c-41e759a8d7ad None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:09.593301 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:09.600118 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b68c58-a90c-439c-8be0-8585a57d1ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:09.601062 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b68c58-a90c-439c-8be0-8585a57d1ed9 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:09.601276 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b68c58-a90c-439c-8be0-8585a57d1ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:09.601921 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b68c58-a90c-439c-8be0-8585a57d1ed9 None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:09.602644 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b68c58-a90c-439c-8be0-8585a57d1ed9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:09.602942 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b68c58-a90c-439c-8be0-8585a57d1ed9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:09.604903 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b68c58-a90c-439c-8be0-8585a57d1ed9 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:09.613584 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 37/71] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 14:12:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:09.614220 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b68c58-a90c-439c-8be0-8585a57d1ed9 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:09.630903 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 35/72] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:09.630903 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2059863-79e0-4987-852d-40d1564be833 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:09.725581 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:09.731583 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b68c58-a90c-439c-8be0-8585a57d1ed9 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:09.739967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2059863-79e0-4987-852d-40d1564be833 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:09.769793 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbace39-f0dd-4c58-91c4-0217f5ae304d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:09.770417 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbace39-f0dd-4c58-91c4-0217f5ae304d None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:09.770987 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbace39-f0dd-4c58-91c4-0217f5ae304d None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:09.770987 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbace39-f0dd-4c58-91c4-0217f5ae304d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:09.781223 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd42e18c-6f6c-4892-aaf9-8c36ff36610d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:09.781911 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd42e18c-6f6c-4892-aaf9-8c36ff36610d None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:09.782063 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbace39-f0dd-4c58-91c4-0217f5ae304d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:09.782382 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd42e18c-6f6c-4892-aaf9-8c36ff36610d None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:09.782834 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd42e18c-6f6c-4892-aaf9-8c36ff36610d None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:09.783634 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd42e18c-6f6c-4892-aaf9-8c36ff36610d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:09.784067 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd42e18c-6f6c-4892-aaf9-8c36ff36610d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:09.785630 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd42e18c-6f6c-4892-aaf9-8c36ff36610d None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:09.787679 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-7bbace39-f0dd-4c58-91c4-0217f5ae304d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:12:09.787679 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:09.787679 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:09.787679 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:09.787679 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:09.787679 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:09.787679 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:09.787679 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:09.787679 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:09.787679 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:09.787679 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:09.787679 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:09.787679 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:09.787679 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:09.787679 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:09.787679 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:09.788501 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:09.788501 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:09.788501 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:09.788501 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:09.788501 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:09.788501 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:09.788501 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:09.788501 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:09.788501 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:09.788501 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:09.788501 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:09.788501 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:09.788501 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:09.788501 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:09.788501 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:09.788501 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:09.789266 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:09.789266 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:09.789266 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:09.789266 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:09.789266 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:09.789266 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:09.789266 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:09.789266 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:09.789266 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:09.789266 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:09.789266 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:12:09.789266 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:12:09.789266 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 38/73] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 14:12:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:12:09.797674 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbace39-f0dd-4c58-91c4-0217f5ae304d None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:09.800128 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 36/74] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:12:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:09.801907 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1a73f-7e92-4749-8f0c-23dab0c82929 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:09.802374 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1a73f-7e92-4749-8f0c-23dab0c82929 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:09.802850 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1a73f-7e92-4749-8f0c-23dab0c82929 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:09.805070 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 37/75] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:09.812010 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1a73f-7e92-4749-8f0c-23dab0c82929 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:09.869432 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:09.922356 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbace39-f0dd-4c58-91c4-0217f5ae304d None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:09.923259 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1a73f-7e92-4749-8f0c-23dab0c82929 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:09.955599 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f7a5f4-3be4-4c4b-99d2-854ba4d02f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:09.956447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f7a5f4-3be4-4c4b-99d2-854ba4d02f5c None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:09.956702 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c1b86-47cc-4396-bf4c-fc92a9d15d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:09.956805 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f7a5f4-3be4-4c4b-99d2-854ba4d02f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:09.956993 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f7a5f4-3be4-4c4b-99d2-854ba4d02f5c None admin] PATH_INFO: `/v3/users/75510b26c21742e091c6eca468df1c06` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:09.957310 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c1b86-47cc-4396-bf4c-fc92a9d15d7c None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:09.957589 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c1b86-47cc-4396-bf4c-fc92a9d15d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:09.957857 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c1b86-47cc-4396-bf4c-fc92a9d15d7c None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:09.958512 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106c1b86-47cc-4396-bf4c-fc92a9d15d7c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:09.958834 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106c1b86-47cc-4396-bf4c-fc92a9d15d7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:09.960211 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106c1b86-47cc-4396-bf4c-fc92a9d15d7c None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:09.967958 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 38/76] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:12:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:09.968548 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c1b86-47cc-4396-bf4c-fc92a9d15d7c None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:09.972916 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f7a5f4-3be4-4c4b-99d2-854ba4d02f5c None admin] RBAC: Authorizing `identity:get_user(user_id=75510b26c21742e091c6eca468df1c06)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:09.974388 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f7a5f4-3be4-4c4b-99d2-854ba4d02f5c None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:09.976061 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 39/77] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:12:09 2022] GET /identity/v3/users/75510b26c21742e091c6eca468df1c06 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:09.977212 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4fa7e-0c7e-4ccc-87d3-05d55724f818 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:09.977497 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4fa7e-0c7e-4ccc-87d3-05d55724f818 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:09.977772 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4fa7e-0c7e-4ccc-87d3-05d55724f818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:09.987193 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:09.996339 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-49e4fa7e-0c7e-4ccc-87d3-05d55724f818 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:09.997543 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-49e4fa7e-0c7e-4ccc-87d3-05d55724f818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:10.063666 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 40/78] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:10.064495 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e4fa7e-0c7e-4ccc-87d3-05d55724f818 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:10.073204 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c1b86-47cc-4396-bf4c-fc92a9d15d7c None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:10.107824 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d81ac-2558-4a8a-807e-844e351e10ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:10.108456 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d81ac-2558-4a8a-807e-844e351e10ce None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:10.108742 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d81ac-2558-4a8a-807e-844e351e10ce None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:10.109017 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d81ac-2558-4a8a-807e-844e351e10ce None admin] PATH_INFO: `/v3/users/a6fda065d398415a9b8308457ab0f0dc` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:10.110165 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d81ac-2558-4a8a-807e-844e351e10ce None admin] RBAC: Authorizing `identity:get_user(user_id=a6fda065d398415a9b8308457ab0f0dc)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:10.111655 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d81ac-2558-4a8a-807e-844e351e10ce None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:10.117851 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 39/79] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:12:09 2022] GET /identity/v3/users/a6fda065d398415a9b8308457ab0f0dc => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:10.118001 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:10.118078 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d81ac-2558-4a8a-807e-844e351e10ce None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:10.176679 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e4fa7e-0c7e-4ccc-87d3-05d55724f818 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:10.200684 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:10.211292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217dacc-1c41-4fc5-9974-d9edc6752f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:10.212045 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217dacc-1c41-4fc5-9974-d9edc6752f46 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:10.212327 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217dacc-1c41-4fc5-9974-d9edc6752f46 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:10.212739 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217dacc-1c41-4fc5-9974-d9edc6752f46 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:10.221100 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d81ac-2558-4a8a-807e-844e351e10ce None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:10.223288 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9217dacc-1c41-4fc5-9974-d9edc6752f46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:10.228857 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-9217dacc-1c41-4fc5-9974-d9edc6752f46 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:12:10.228857 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:10.228857 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:10.228857 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:10.228857 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:10.228857 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:10.228857 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:10.228857 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:10.228857 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:10.228857 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:10.228857 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:10.228857 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:10.228857 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:10.228857 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:10.228857 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:10.228857 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:10.229765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:10.229765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:10.229765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:10.229765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:10.229765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:10.229765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:10.229765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:10.229765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:10.229765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:10.229765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:10.229765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:10.229765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:10.229765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:10.229765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:10.229765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:10.229765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:10.235245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:10.235245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:10.235245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:10.235245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:10.235245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:10.235245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:10.235245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:10.235245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:10.235245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:10.235245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:10.235245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:12:10.235245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:12:10.235245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 41/80] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:12:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:12:10.235245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217dacc-1c41-4fc5-9974-d9edc6752f46 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:10.258354 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85678fc-9be5-4b1c-92c0-23066dce4ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:10.259025 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85678fc-9be5-4b1c-92c0-23066dce4ecf None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:10.259310 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85678fc-9be5-4b1c-92c0-23066dce4ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:10.259588 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85678fc-9be5-4b1c-92c0-23066dce4ecf None admin] PATH_INFO: `/v3/users/75510b26c21742e091c6eca468df1c06` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:10.260756 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85678fc-9be5-4b1c-92c0-23066dce4ecf None admin] RBAC: Authorizing `identity:get_user(user_id=75510b26c21742e091c6eca468df1c06)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:10.262206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85678fc-9be5-4b1c-92c0-23066dce4ecf None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:10.263948 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 40/81] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:12:10 2022] GET /identity/v3/users/75510b26c21742e091c6eca468df1c06 => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:10.264545 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85678fc-9be5-4b1c-92c0-23066dce4ecf None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:10.344433 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:10.347602 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217dacc-1c41-4fc5-9974-d9edc6752f46 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:10.371279 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85678fc-9be5-4b1c-92c0-23066dce4ecf None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:10.382529 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0394c80d-95a5-4f60-9287-fdf25e3857ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:10.383162 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394c80d-95a5-4f60-9287-fdf25e3857ac None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:10.383446 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394c80d-95a5-4f60-9287-fdf25e3857ac None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:10.383721 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394c80d-95a5-4f60-9287-fdf25e3857ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:10.404431 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3fa5b3-3932-4ff9-a535-d2550426ecbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:10.405369 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3fa5b3-3932-4ff9-a535-d2550426ecbb None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:10.405648 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3fa5b3-3932-4ff9-a535-d2550426ecbb None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:10.405926 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3fa5b3-3932-4ff9-a535-d2550426ecbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:10.423742 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:10.452393 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0394c80d-95a5-4f60-9287-fdf25e3857ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:10.453825 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0394c80d-95a5-4f60-9287-fdf25e3857ac None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:10.472670 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3fa5b3-3932-4ff9-a535-d2550426ecbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:10.474129 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3fa5b3-3932-4ff9-a535-d2550426ecbb None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:10.547503 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:10.551538 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 42/82] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:12:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:12:10.552509 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0394c80d-95a5-4f60-9287-fdf25e3857ac None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:10.574553 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 41/83] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:12:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:12:10.575443 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3fa5b3-3932-4ff9-a535-d2550426ecbb None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:10.622755 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:10.649819 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0394c80d-95a5-4f60-9287-fdf25e3857ac None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:10.673657 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3fa5b3-3932-4ff9-a535-d2550426ecbb None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:10.683679 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f369a4d-6868-4df9-bcee-6ef0b0cf315d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:10.684282 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f369a4d-6868-4df9-bcee-6ef0b0cf315d None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:10.684566 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f369a4d-6868-4df9-bcee-6ef0b0cf315d None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:10.684831 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f369a4d-6868-4df9-bcee-6ef0b0cf315d None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:10.685486 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f369a4d-6868-4df9-bcee-6ef0b0cf315d None admin] RBAC: Adding query filter params (name=member) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:10.685808 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f369a4d-6868-4df9-bcee-6ef0b0cf315d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:10.687220 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f369a4d-6868-4df9-bcee-6ef0b0cf315d None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:10.695062 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 43/84] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 14:12:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:10.695839 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f369a4d-6868-4df9-bcee-6ef0b0cf315d None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:10.708988 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f30cd82-065e-4236-b9ea-6b7d6bebd651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:10.709541 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30cd82-065e-4236-b9ea-6b7d6bebd651 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:10.709820 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30cd82-065e-4236-b9ea-6b7d6bebd651 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:10.710100 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30cd82-065e-4236-b9ea-6b7d6bebd651 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:10.755355 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:10.778615 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f30cd82-065e-4236-b9ea-6b7d6bebd651 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:10.780380 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f30cd82-065e-4236-b9ea-6b7d6bebd651 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:10.794033 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f369a4d-6868-4df9-bcee-6ef0b0cf315d None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:10.828100 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:10.830362 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8259f07-0892-43e3-827b-f476d8e1e386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:10.831131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8259f07-0892-43e3-827b-f476d8e1e386 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:10.831374 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8259f07-0892-43e3-827b-f476d8e1e386 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:10.831647 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8259f07-0892-43e3-827b-f476d8e1e386 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:10.832762 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8259f07-0892-43e3-827b-f476d8e1e386 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:10.834213 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8259f07-0892-43e3-827b-f476d8e1e386 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:10.835805 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 44/85] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:12:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:10.836559 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8259f07-0892-43e3-827b-f476d8e1e386 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:10.884483 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 42/86] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:12:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:12:10.885498 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f30cd82-065e-4236-b9ea-6b7d6bebd651 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:10.942586 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8259f07-0892-43e3-827b-f476d8e1e386 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:10.972100 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:10.978905 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a0cc35-37b3-43e4-9c51-0fb6decd1880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:10.979538 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0cc35-37b3-43e4-9c51-0fb6decd1880 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:10.979820 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0cc35-37b3-43e4-9c51-0fb6decd1880 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:10.980097 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0cc35-37b3-43e4-9c51-0fb6decd1880 None admin] PATH_INFO: `/v3/projects/a7eea203d96a471cb590c418bb060d23` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:10.981212 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a0cc35-37b3-43e4-9c51-0fb6decd1880 None admin] RBAC: Authorizing `identity:get_project(project_id=a7eea203d96a471cb590c418bb060d23)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:10.982677 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a0cc35-37b3-43e4-9c51-0fb6decd1880 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:10.984364 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 45/87] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:12:10 2022] GET /identity/v3/projects/a7eea203d96a471cb590c418bb060d23 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:10.984890 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f30cd82-065e-4236-b9ea-6b7d6bebd651 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:10.985123 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a0cc35-37b3-43e4-9c51-0fb6decd1880 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:11.020910 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3157412-7593-4c2c-bf60-7320a8b9ff4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:11.021543 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3157412-7593-4c2c-bf60-7320a8b9ff4b None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:11.021824 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3157412-7593-4c2c-bf60-7320a8b9ff4b None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:11.022100 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3157412-7593-4c2c-bf60-7320a8b9ff4b None admin] PATH_INFO: `/v3/projects/a7eea203d96a471cb590c418bb060d23` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:11.023255 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3157412-7593-4c2c-bf60-7320a8b9ff4b None admin] RBAC: Authorizing `identity:get_project(project_id=a7eea203d96a471cb590c418bb060d23)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:11.024704 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3157412-7593-4c2c-bf60-7320a8b9ff4b None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:11.026979 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 43/88] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:12:10 2022] GET /identity/v3/projects/a7eea203d96a471cb590c418bb060d23 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:11.027660 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3157412-7593-4c2c-bf60-7320a8b9ff4b None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:11.047587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:11.084548 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a0cc35-37b3-43e4-9c51-0fb6decd1880 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:11.117879 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b72d4-ca0d-49e3-b12b-cc7619612fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:11.118512 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b72d4-ca0d-49e3-b12b-cc7619612fb6 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:11.119335 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b72d4-ca0d-49e3-b12b-cc7619612fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:11.119611 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b72d4-ca0d-49e3-b12b-cc7619612fb6 None admin] PATH_INFO: `/v3/users/75510b26c21742e091c6eca468df1c06` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:11.120736 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82b72d4-ca0d-49e3-b12b-cc7619612fb6 None admin] RBAC: Authorizing `identity:get_user(user_id=75510b26c21742e091c6eca468df1c06)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:11.122195 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82b72d4-ca0d-49e3-b12b-cc7619612fb6 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:11.123996 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 46/89] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:12:10 2022] GET /identity/v3/users/75510b26c21742e091c6eca468df1c06 => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:11.124558 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b72d4-ca0d-49e3-b12b-cc7619612fb6 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:11.132075 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3157412-7593-4c2c-bf60-7320a8b9ff4b None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:11.166053 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cde3e2-4456-461d-bf38-fee0539f511a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:11.166662 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cde3e2-4456-461d-bf38-fee0539f511a None admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:11.166972 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cde3e2-4456-461d-bf38-fee0539f511a None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:11.167246 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cde3e2-4456-461d-bf38-fee0539f511a None admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:11.167897 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cde3e2-4456-461d-bf38-fee0539f511a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:11.169326 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cde3e2-4456-461d-bf38-fee0539f511a None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:11.191193 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 44/90] 10.222.0.52 () {60 vars in 1132 bytes} [Sat Sep 24 14:12:11 2022] POST /identity/v3/users => generated 289 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:12:11.192007 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cde3e2-4456-461d-bf38-fee0539f511a None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:11.192881 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:11.233138 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b72d4-ca0d-49e3-b12b-cc7619612fb6 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:11.280696 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e41a7c0-e08a-48ba-ad65-83f6f0211f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:11.281474 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41a7c0-e08a-48ba-ad65-83f6f0211f7a None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:11.281781 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41a7c0-e08a-48ba-ad65-83f6f0211f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:11.282144 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41a7c0-e08a-48ba-ad65-83f6f0211f7a None admin] PATH_INFO: `/v3/projects/a7eea203d96a471cb590c418bb060d23` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:11.283429 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e41a7c0-e08a-48ba-ad65-83f6f0211f7a None admin] RBAC: Authorizing `identity:get_project(project_id=a7eea203d96a471cb590c418bb060d23)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:11.285070 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e41a7c0-e08a-48ba-ad65-83f6f0211f7a None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:11.292188 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 47/91] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:12:11 2022] GET /identity/v3/projects/a7eea203d96a471cb590c418bb060d23 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:11.292188 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e41a7c0-e08a-48ba-ad65-83f6f0211f7a None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:11.299062 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:11.337122 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cde3e2-4456-461d-bf38-fee0539f511a None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:11.379189 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834e920-ab00-4494-b58b-df6ce213b919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:11.379817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834e920-ab00-4494-b58b-df6ce213b919 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:11.380894 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834e920-ab00-4494-b58b-df6ce213b919 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:11.381179 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834e920-ab00-4494-b58b-df6ce213b919 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:11.381918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2834e920-ab00-4494-b58b-df6ce213b919 None admin] RBAC: Adding query filter params (role.id=640af5182e6848498c16a19cddbfb738, scope.project.id=a7eea203d96a471cb590c418bb060d23, user.id=75510b26c21742e091c6eca468df1c06) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:11.382230 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2834e920-ab00-4494-b58b-df6ce213b919 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:11.383716 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2834e920-ab00-4494-b58b-df6ce213b919 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:11.391650 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 45/92] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:12:11 2022] GET /identity/v3/role_assignments?user.id=75510b26c21742e091c6eca468df1c06&scope.project.id=a7eea203d96a471cb590c418bb060d23&role.id=640af5182e6848498c16a19cddbfb738 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:11.392302 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834e920-ab00-4494-b58b-df6ce213b919 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:11.406143 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e41a7c0-e08a-48ba-ad65-83f6f0211f7a None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:11.455382 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15bfe9-3811-4fc3-8b1d-28252c927866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:11.456504 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15bfe9-3811-4fc3-8b1d-28252c927866 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:11.456504 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15bfe9-3811-4fc3-8b1d-28252c927866 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:11.456690 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15bfe9-3811-4fc3-8b1d-28252c927866 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:11.457309 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c15bfe9-3811-4fc3-8b1d-28252c927866 None admin] RBAC: Adding query filter params (role.id=d00b78c32dbf4cc7925a55bb621d8fcf, scope.project.id=a7eea203d96a471cb590c418bb060d23, user.id=a6fda065d398415a9b8308457ab0f0dc) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:11.457625 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c15bfe9-3811-4fc3-8b1d-28252c927866 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:11.459194 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c15bfe9-3811-4fc3-8b1d-28252c927866 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:11.461630 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:11.467701 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 48/93] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:12:11 2022] GET /identity/v3/role_assignments?user.id=a6fda065d398415a9b8308457ab0f0dc&scope.project.id=a7eea203d96a471cb590c418bb060d23&role.id=d00b78c32dbf4cc7925a55bb621d8fcf => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:11.469716 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15bfe9-3811-4fc3-8b1d-28252c927866 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:11.550178 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834e920-ab00-4494-b58b-df6ce213b919 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:11.609134 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8482197-16f9-4219-9797-4eb4b9894ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:11.609772 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8482197-16f9-4219-9797-4eb4b9894ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:11.610160 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8482197-16f9-4219-9797-4eb4b9894ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:11.610441 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8482197-16f9-4219-9797-4eb4b9894ca4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:11.616219 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:11.632287 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15bfe9-3811-4fc3-8b1d-28252c927866 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:11.679109 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c63ec5-a2f7-4a40-b696-17d19c311ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:11.679109 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c63ec5-a2f7-4a40-b696-17d19c311ce7 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:11.679478 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c63ec5-a2f7-4a40-b696-17d19c311ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:11.679742 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c63ec5-a2f7-4a40-b696-17d19c311ce7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:11.680486 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c63ec5-a2f7-4a40-b696-17d19c311ce7 None admin] RBAC: Adding query filter params (role.id=25257d00ad2640c2a57d257b0279155f, scope.project.id=a7eea203d96a471cb590c418bb060d23, user.id=75510b26c21742e091c6eca468df1c06) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:11.680796 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c63ec5-a2f7-4a40-b696-17d19c311ce7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:11.683795 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c63ec5-a2f7-4a40-b696-17d19c311ce7 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:11.689465 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 49/94] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:12:11 2022] GET /identity/v3/role_assignments?user.id=75510b26c21742e091c6eca468df1c06&scope.project.id=a7eea203d96a471cb590c418bb060d23&role.id=25257d00ad2640c2a57d257b0279155f => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:11.709833 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8482197-16f9-4219-9797-4eb4b9894ca4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:11.711648 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8482197-16f9-4219-9797-4eb4b9894ca4 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:11.835760 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 46/95] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:12:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:12:11.841557 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c63ec5-a2f7-4a40-b696-17d19c311ce7 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:11.913657 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:11.958440 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c63ec5-a2f7-4a40-b696-17d19c311ce7 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:11.996079 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ade2a-a7f0-4efb-877b-c6ee96eec9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:11.996727 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ade2a-a7f0-4efb-877b-c6ee96eec9af None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:11.997011 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ade2a-a7f0-4efb-877b-c6ee96eec9af None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:11.997286 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ade2a-a7f0-4efb-877b-c6ee96eec9af None admin] PATH_INFO: `/v3/projects/f7af21ee123e483ea1be1cb0e7f9f73b` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:11.998413 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5ade2a-a7f0-4efb-877b-c6ee96eec9af None admin] RBAC: Authorizing `identity:get_project(project_id=f7af21ee123e483ea1be1cb0e7f9f73b)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:11.999915 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5ade2a-a7f0-4efb-877b-c6ee96eec9af None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:12.001861 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 50/96] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:12:11 2022] GET /identity/v3/projects/f7af21ee123e483ea1be1cb0e7f9f73b => generated 312 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:12.007809 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8482197-16f9-4219-9797-4eb4b9894ca4 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:12.044054 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:12.121864 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8482197-16f9-4219-9797-4eb4b9894ca4 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:12.157912 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7934455b-7cc0-4042-ab0c-a5209827644e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:12.158530 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934455b-7cc0-4042-ab0c-a5209827644e None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:12.158829 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934455b-7cc0-4042-ab0c-a5209827644e None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:12.159112 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934455b-7cc0-4042-ab0c-a5209827644e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:12.159840 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7934455b-7cc0-4042-ab0c-a5209827644e None admin] RBAC: Adding query filter params (role.id=640af5182e6848498c16a19cddbfb738, scope.project.id=f7af21ee123e483ea1be1cb0e7f9f73b, user.id=75510b26c21742e091c6eca468df1c06) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:12.160149 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7934455b-7cc0-4042-ab0c-a5209827644e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:12.161613 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7934455b-7cc0-4042-ab0c-a5209827644e None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:12.168357 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 47/97] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:12:12 2022] GET /identity/v3/role_assignments?user.id=75510b26c21742e091c6eca468df1c06&scope.project.id=f7af21ee123e483ea1be1cb0e7f9f73b&role.id=640af5182e6848498c16a19cddbfb738 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:14.320992 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e544f1e-37c5-4021-b01b-6eb7260c5a94 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:14.321574 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e544f1e-37c5-4021-b01b-6eb7260c5a94 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:14.321574 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e544f1e-37c5-4021-b01b-6eb7260c5a94 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:14.322952 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 51/98] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:14.336386 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33129c42-bc21-4fd7-b0d2-b4315e74820a None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:14.336880 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33129c42-bc21-4fd7-b0d2-b4315e74820a None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:14.336949 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33129c42-bc21-4fd7-b0d2-b4315e74820a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:14.364511 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-33129c42-bc21-4fd7-b0d2-b4315e74820a None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:14.365747 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-33129c42-bc21-4fd7-b0d2-b4315e74820a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:14.453275 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 48/99] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:14.464332 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f852f26-6b66-4996-8ee6-dd9a188f4616 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:14.464843 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f852f26-6b66-4996-8ee6-dd9a188f4616 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:14.465333 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f852f26-6b66-4996-8ee6-dd9a188f4616 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:14.466870 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 52/100] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:14.472027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774edad-aafc-4df8-9620-9d29f83cb4b1 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:14.472323 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774edad-aafc-4df8-9620-9d29f83cb4b1 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:14.472601 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774edad-aafc-4df8-9620-9d29f83cb4b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:14.498208 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-b774edad-aafc-4df8-9620-9d29f83cb4b1 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:14.499798 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-b774edad-aafc-4df8-9620-9d29f83cb4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:14.578036 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:14.633605 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 49/101] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:14.758440 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c9beb-ed83-4347-b532-80d6447b6766 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:14.759085 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c9beb-ed83-4347-b532-80d6447b6766 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:14.760096 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c9beb-ed83-4347-b532-80d6447b6766 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:14.761684 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 53/102] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:14.763376 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37161ed3-d5cf-4673-af3e-1528e4324b0e None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:14.763856 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37161ed3-d5cf-4673-af3e-1528e4324b0e None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:14.764316 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37161ed3-d5cf-4673-af3e-1528e4324b0e None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:14.766775 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83458d1a-0fbe-4a7a-ac3d-fc723dda0e0f None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:14.767284 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83458d1a-0fbe-4a7a-ac3d-fc723dda0e0f None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:14.767748 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83458d1a-0fbe-4a7a-ac3d-fc723dda0e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:14.774231 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 50/103] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:14 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:14.784179 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd46901-41fd-4c40-982b-694f9a89f8b2 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:14.784745 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd46901-41fd-4c40-982b-694f9a89f8b2 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:14.785210 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd46901-41fd-4c40-982b-694f9a89f8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:14.798785 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-83458d1a-0fbe-4a7a-ac3d-fc723dda0e0f None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:14.805226 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-83458d1a-0fbe-4a7a-ac3d-fc723dda0e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:14.823854 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-7bd46901-41fd-4c40-982b-694f9a89f8b2 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:14.825366 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-7bd46901-41fd-4c40-982b-694f9a89f8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:14.936894 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 54/104] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:12:14.937750 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83458d1a-0fbe-4a7a-ac3d-fc723dda0e0f None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:14.940557 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 51/105] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:14.999559 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:15.042890 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83458d1a-0fbe-4a7a-ac3d-fc723dda0e0f None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:15.070064 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd46901-41fd-4c40-982b-694f9a89f8b2 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:15.078859 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec8d9a-d7c2-4ef9-9b09-b3a3dfc01c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:15.079511 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec8d9a-d7c2-4ef9-9b09-b3a3dfc01c89 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:15.079848 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec8d9a-d7c2-4ef9-9b09-b3a3dfc01c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:15.080130 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec8d9a-d7c2-4ef9-9b09-b3a3dfc01c89 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:15.091341 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ec8d9a-d7c2-4ef9-9b09-b3a3dfc01c89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:15.096681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-43ec8d9a-d7c2-4ef9-9b09-b3a3dfc01c89 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:12:15.096681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:15.096681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:15.096681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:15.096681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:15.096681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:15.096681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:15.096681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:15.096681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:15.096681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:15.096681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:15.096681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:15.096681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:15.096681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:15.096681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:15.096681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:15.097511 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:15.097511 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:15.097511 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:15.097511 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:15.097511 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:15.097511 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:15.097511 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:15.097511 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:15.097511 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:15.097511 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:15.097511 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:15.097511 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:15.097511 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:15.097511 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:15.097511 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:15.097511 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:15.098242 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:15.098242 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:15.098242 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:15.098242 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:15.098242 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:15.098242 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:15.098242 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:15.098242 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:15.098242 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:15.098242 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:15.098242 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:12:15.098242 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:12:15.098242 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 55/106] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:12:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:12:15.109139 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec8d9a-d7c2-4ef9-9b09-b3a3dfc01c89 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:15.224665 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec8d9a-d7c2-4ef9-9b09-b3a3dfc01c89 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:15.255183 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:15.270579 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f6864-1bd4-461d-aa3a-eb636c3630c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:15.271238 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f6864-1bd4-461d-aa3a-eb636c3630c5 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:15.271695 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f6864-1bd4-461d-aa3a-eb636c3630c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:15.271984 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f6864-1bd4-461d-aa3a-eb636c3630c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:15.272664 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94f6864-1bd4-461d-aa3a-eb636c3630c5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:15.272981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94f6864-1bd4-461d-aa3a-eb636c3630c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:15.274479 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94f6864-1bd4-461d-aa3a-eb636c3630c5 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:15.285175 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 56/107] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 14:12:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:15.291370 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f6864-1bd4-461d-aa3a-eb636c3630c5 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:15.311832 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:15.344028 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd46901-41fd-4c40-982b-694f9a89f8b2 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:15.380758 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0655758-0039-4040-b012-7576d1a68701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:15.381395 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0655758-0039-4040-b012-7576d1a68701 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:15.381675 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0655758-0039-4040-b012-7576d1a68701 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:15.381948 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0655758-0039-4040-b012-7576d1a68701 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:15.393765 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0655758-0039-4040-b012-7576d1a68701 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:15.399307 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-e0655758-0039-4040-b012-7576d1a68701 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:12:15.399307 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:15.399307 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:15.399307 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:15.399307 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:15.399307 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:15.399307 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:15.399307 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:15.399307 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:15.399307 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:15.399307 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:15.399307 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:15.399307 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:15.399307 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:15.399307 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:15.399307 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:15.400726 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:15.400726 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:15.400726 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:15.400726 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:15.400726 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:15.400726 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:15.400726 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:15.400726 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:15.400726 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:15.400726 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:15.400726 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:15.400726 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:15.400726 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:15.400726 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:15.400726 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:15.400726 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:15.401522 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:15.401522 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:15.401522 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:15.401522 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:15.401522 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:15.401522 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:15.401522 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:15.401522 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:15.401522 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:15.401522 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:15.401522 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:12:15.401522 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:12:15.401522 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 52/108] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 14:12:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:12:15.402169 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138996a7-014b-4dbd-9f01-ccdbeb8ba2f9 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:15.402858 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138996a7-014b-4dbd-9f01-ccdbeb8ba2f9 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:15.402858 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138996a7-014b-4dbd-9f01-ccdbeb8ba2f9 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:15.408687 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 53/109] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:15.409715 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138996a7-014b-4dbd-9f01-ccdbeb8ba2f9 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:15.411827 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f6864-1bd4-461d-aa3a-eb636c3630c5 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:15.450496 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538e1a2-4db8-4ed4-813b-652e8a1ae987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:15.451159 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538e1a2-4db8-4ed4-813b-652e8a1ae987 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:15.451440 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538e1a2-4db8-4ed4-813b-652e8a1ae987 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:15.451710 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538e1a2-4db8-4ed4-813b-652e8a1ae987 None admin] PATH_INFO: `/v3/users/75510b26c21742e091c6eca468df1c06` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:15.452820 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c538e1a2-4db8-4ed4-813b-652e8a1ae987 None admin] RBAC: Authorizing `identity:get_user(user_id=75510b26c21742e091c6eca468df1c06)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:15.454264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c538e1a2-4db8-4ed4-813b-652e8a1ae987 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:15.455871 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 57/110] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:12:15 2022] GET /identity/v3/users/75510b26c21742e091c6eca468df1c06 => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:15.459151 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538e1a2-4db8-4ed4-813b-652e8a1ae987 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:15.515963 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:15.527320 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138996a7-014b-4dbd-9f01-ccdbeb8ba2f9 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:15.568060 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76755d8-d95f-4c5a-aa72-50cd250d0610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:15.568987 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76755d8-d95f-4c5a-aa72-50cd250d0610 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:15.569283 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76755d8-d95f-4c5a-aa72-50cd250d0610 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:15.569560 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76755d8-d95f-4c5a-aa72-50cd250d0610 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:15.570669 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76755d8-d95f-4c5a-aa72-50cd250d0610 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:15.571811 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538e1a2-4db8-4ed4-813b-652e8a1ae987 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:15.572806 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76755d8-d95f-4c5a-aa72-50cd250d0610 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:15.574399 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 54/111] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:12:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:15.575541 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae8fa9b-6334-445f-8d8c-33f1e05e2905 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:15.575823 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae8fa9b-6334-445f-8d8c-33f1e05e2905 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:15.576099 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae8fa9b-6334-445f-8d8c-33f1e05e2905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:15.595381 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-9ae8fa9b-6334-445f-8d8c-33f1e05e2905 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:15.596485 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-9ae8fa9b-6334-445f-8d8c-33f1e05e2905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:15.611141 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:15.614989 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6a13e-308a-4701-a34f-439988c31363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:15.615762 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6a13e-308a-4701-a34f-439988c31363 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:15.615866 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6a13e-308a-4701-a34f-439988c31363 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:15.616150 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6a13e-308a-4701-a34f-439988c31363 None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:15.616898 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe6a13e-308a-4701-a34f-439988c31363 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:15.617122 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe6a13e-308a-4701-a34f-439988c31363 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:15.618582 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe6a13e-308a-4701-a34f-439988c31363 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:15.626974 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 58/112] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:12:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:15.627811 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6a13e-308a-4701-a34f-439988c31363 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:15.678381 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 55/113] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:15.688017 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae8fa9b-6334-445f-8d8c-33f1e05e2905 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:15.742315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6a13e-308a-4701-a34f-439988c31363 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:15.758932 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:15.784660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb3f5c9-50cd-410a-bd21-87b7d9963ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:15.785475 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3f5c9-50cd-410a-bd21-87b7d9963ecc None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:15.786107 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3f5c9-50cd-410a-bd21-87b7d9963ecc None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:15.786312 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3f5c9-50cd-410a-bd21-87b7d9963ecc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:15.800506 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb3f5c9-50cd-410a-bd21-87b7d9963ecc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:15.807087 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-2cb3f5c9-50cd-410a-bd21-87b7d9963ecc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 14:12:15.807087 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:15.807087 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:15.807087 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:15.807087 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:15.807087 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:15.807087 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:15.807087 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:15.807087 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:15.807087 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:15.807087 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:15.807087 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:15.807087 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:15.807087 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:15.807087 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:15.807087 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:15.808227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:15.808227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:15.808227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:15.808227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:15.808227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:15.808227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:15.808227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:15.808227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:15.808227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:15.808227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:15.808227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:15.808227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:15.808227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:15.808227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:15.808227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:15.808227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:15.809185 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:15.809185 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:15.809185 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:15.809185 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:15.809185 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:15.809185 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:15.809185 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:15.809185 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:15.809185 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:15.809185 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:15.809185 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 14:12:15.809185 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:12:15.810940 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 59/114] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:12:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:12:15.811887 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb3f5c9-50cd-410a-bd21-87b7d9963ecc None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:15.812750 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae8fa9b-6334-445f-8d8c-33f1e05e2905 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:15.862886 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e4e464-d17c-4f11-9352-081977614c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:15.864003 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4e464-d17c-4f11-9352-081977614c56 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:15.864421 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4e464-d17c-4f11-9352-081977614c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:15.865289 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4e464-d17c-4f11-9352-081977614c56 None admin] PATH_INFO: `/v3/projects/a7eea203d96a471cb590c418bb060d23` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:15.866750 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e4e464-d17c-4f11-9352-081977614c56 None admin] RBAC: Authorizing `identity:get_project(project_id=a7eea203d96a471cb590c418bb060d23)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:15.868623 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e4e464-d17c-4f11-9352-081977614c56 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:15.870963 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 56/115] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:12:15 2022] GET /identity/v3/projects/a7eea203d96a471cb590c418bb060d23 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:15.871889 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e4e464-d17c-4f11-9352-081977614c56 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:15.893157 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:15.928063 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb3f5c9-50cd-410a-bd21-87b7d9963ecc None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:15.962569 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d01707-a7fe-4ebd-867a-29e7d129f5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:15.963699 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d01707-a7fe-4ebd-867a-29e7d129f5a4 None admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:15.963980 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d01707-a7fe-4ebd-867a-29e7d129f5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:15.964253 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d01707-a7fe-4ebd-867a-29e7d129f5a4 None admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:15.964909 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d01707-a7fe-4ebd-867a-29e7d129f5a4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:15.966372 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d01707-a7fe-4ebd-867a-29e7d129f5a4 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:15.988400 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e4e464-d17c-4f11-9352-081977614c56 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:15.995249 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 60/116] 10.222.0.52 () {60 vars in 1141 bytes} [Sat Sep 24 14:12:15 2022] POST /identity/v3/projects => generated 302 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:12:16.004056 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d01707-a7fe-4ebd-867a-29e7d129f5a4 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:16.029453 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d16439-9809-47e5-b41c-8b0e41047296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:16.030362 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d16439-9809-47e5-b41c-8b0e41047296 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:16.030641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d16439-9809-47e5-b41c-8b0e41047296 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:16.030954 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d16439-9809-47e5-b41c-8b0e41047296 None admin] PATH_INFO: `/v3/users/a6fda065d398415a9b8308457ab0f0dc` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:16.032169 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d16439-9809-47e5-b41c-8b0e41047296 None admin] RBAC: Authorizing `identity:get_user(user_id=a6fda065d398415a9b8308457ab0f0dc)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:16.033643 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d16439-9809-47e5-b41c-8b0e41047296 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:16.034929 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:16.035551 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 57/117] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:12:15 2022] GET /identity/v3/users/a6fda065d398415a9b8308457ab0f0dc => generated 247 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:16.036222 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d16439-9809-47e5-b41c-8b0e41047296 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:16.128875 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d01707-a7fe-4ebd-867a-29e7d129f5a4 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:16.141051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:16.159769 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d16439-9809-47e5-b41c-8b0e41047296 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:16.168551 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee56651-fd29-4cae-a823-83c3d2e7f405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:16.168887 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee56651-fd29-4cae-a823-83c3d2e7f405 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:16.169167 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee56651-fd29-4cae-a823-83c3d2e7f405 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:16.169443 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee56651-fd29-4cae-a823-83c3d2e7f405 None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:16.170111 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee56651-fd29-4cae-a823-83c3d2e7f405 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:16.170416 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee56651-fd29-4cae-a823-83c3d2e7f405 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:16.171857 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee56651-fd29-4cae-a823-83c3d2e7f405 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:16.181319 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 61/118] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 14:12:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:16.182581 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae0689-6cb0-4eb6-b4e1-50fd5f85a5c8 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:16.182901 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae0689-6cb0-4eb6-b4e1-50fd5f85a5c8 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:16.183174 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae0689-6cb0-4eb6-b4e1-50fd5f85a5c8 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:16.184387 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 62/119] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:16.185141 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdae0689-6cb0-4eb6-b4e1-50fd5f85a5c8 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:16.204878 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d375a6e7-217d-4376-ab6c-e243cea7c5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:16.205863 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375a6e7-217d-4376-ab6c-e243cea7c5ff None admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:16.206169 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375a6e7-217d-4376-ab6c-e243cea7c5ff None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:16.206458 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375a6e7-217d-4376-ab6c-e243cea7c5ff None admin] PATH_INFO: `/v3/projects/a7eea203d96a471cb590c418bb060d23/users/75510b26c21742e091c6eca468df1c06/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:16.209463 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d375a6e7-217d-4376-ab6c-e243cea7c5ff None admin] RBAC: Authorizing `identity:create_grant(project_id=a7eea203d96a471cb590c418bb060d23, user_id=75510b26c21742e091c6eca468df1c06, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:16.210995 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d375a6e7-217d-4376-ab6c-e243cea7c5ff None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:16.229875 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 58/120] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 14:12:16 2022] PUT /identity/v3/projects/a7eea203d96a471cb590c418bb060d23/users/75510b26c21742e091c6eca468df1c06/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:12:16.230746 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d375a6e7-217d-4376-ab6c-e243cea7c5ff None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:16.313625 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:16.335739 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdae0689-6cb0-4eb6-b4e1-50fd5f85a5c8 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:16.364991 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d375a6e7-217d-4376-ab6c-e243cea7c5ff None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:16.378331 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a76506-d680-4b36-ba98-1c15ccbe98e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:16.379832 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a76506-d680-4b36-ba98-1c15ccbe98e9 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:16.380398 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a76506-d680-4b36-ba98-1c15ccbe98e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:16.380996 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a76506-d680-4b36-ba98-1c15ccbe98e9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:16.395714 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a76506-d680-4b36-ba98-1c15ccbe98e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:16.401604 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-b0a76506-d680-4b36-ba98-1c15ccbe98e9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:12:16.401604 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:16.401604 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:16.401604 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:16.401604 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:16.401604 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:16.401604 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:16.401604 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:16.401604 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:16.401604 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:16.401604 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:16.401604 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:16.401604 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:16.401604 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:16.401604 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:16.401604 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:16.402518 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:16.402518 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:16.402518 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:16.402518 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:16.402518 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:16.402518 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:16.402518 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:16.402518 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:16.402518 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:16.402518 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:16.402518 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:16.402518 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:16.402518 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:16.402518 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:16.402518 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:16.402518 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:16.403659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:16.403659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:16.403659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:16.403659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:16.403659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:16.403659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:16.403659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:16.403659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:16.403659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:16.403659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:16.403659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:12:16.403659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:12:16.406091 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 63/121] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 14:12:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:12:16.409479 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a76506-d680-4b36-ba98-1c15ccbe98e9 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:16.415547 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5925ea-b49a-452c-9c33-546aec09efe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:16.416279 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5925ea-b49a-452c-9c33-546aec09efe7 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:16.416561 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5925ea-b49a-452c-9c33-546aec09efe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:16.416834 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5925ea-b49a-452c-9c33-546aec09efe7 None admin] PATH_INFO: `/v3/projects/a7eea203d96a471cb590c418bb060d23` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:16.418147 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5925ea-b49a-452c-9c33-546aec09efe7 None admin] RBAC: Authorizing `identity:get_project(project_id=a7eea203d96a471cb590c418bb060d23)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:16.419793 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5925ea-b49a-452c-9c33-546aec09efe7 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:16.421590 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 59/122] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:12:16 2022] GET /identity/v3/projects/a7eea203d96a471cb590c418bb060d23 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:16.422879 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67d72e-8561-481c-b2ca-2557d50404cb None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:16.423273 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67d72e-8561-481c-b2ca-2557d50404cb None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:16.423653 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67d72e-8561-481c-b2ca-2557d50404cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:16.443577 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-2c67d72e-8561-481c-b2ca-2557d50404cb None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:16.444950 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-2c67d72e-8561-481c-b2ca-2557d50404cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:16.455119 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:16.538403 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a76506-d680-4b36-ba98-1c15ccbe98e9 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:16.539263 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 60/123] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:16.540241 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67d72e-8561-481c-b2ca-2557d50404cb None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:16.577263 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c9c6f4-4444-4136-8a62-618ee5a7aa8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:16.578060 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9c6f4-4444-4136-8a62-618ee5a7aa8a None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:16.579164 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9c6f4-4444-4136-8a62-618ee5a7aa8a None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:16.579164 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9c6f4-4444-4136-8a62-618ee5a7aa8a None admin] PATH_INFO: `/v3/users/44e73f0cd35648198c204a3f657edccb` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:16.594047 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c9c6f4-4444-4136-8a62-618ee5a7aa8a None admin] RBAC: Authorizing `identity:get_user(user_id=44e73f0cd35648198c204a3f657edccb)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:16.596347 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c9c6f4-4444-4136-8a62-618ee5a7aa8a None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:16.598125 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 64/124] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:12:16 2022] GET /identity/v3/users/44e73f0cd35648198c204a3f657edccb => generated 289 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:16.599501 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c9c6f4-4444-4136-8a62-618ee5a7aa8a None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:16.608553 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:16.683586 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67d72e-8561-481c-b2ca-2557d50404cb None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:16.723519 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadba86c-37be-4b66-b5db-ec00b6d5ce78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:16.724747 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadba86c-37be-4b66-b5db-ec00b6d5ce78 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:16.725252 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadba86c-37be-4b66-b5db-ec00b6d5ce78 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:16.725724 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadba86c-37be-4b66-b5db-ec00b6d5ce78 None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:16.726604 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadba86c-37be-4b66-b5db-ec00b6d5ce78 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:16.727128 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadba86c-37be-4b66-b5db-ec00b6d5ce78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:16.729389 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadba86c-37be-4b66-b5db-ec00b6d5ce78 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:16.738504 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 61/125] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:12:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:16.739643 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadba86c-37be-4b66-b5db-ec00b6d5ce78 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:16.743696 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:16.824790 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c9c6f4-4444-4136-8a62-618ee5a7aa8a None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:16.872248 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadba86c-37be-4b66-b5db-ec00b6d5ce78 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:16.874142 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca98c8f-42df-490c-ac88-b675e5e0defd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:16.874841 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca98c8f-42df-490c-ac88-b675e5e0defd None admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:16.876140 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca98c8f-42df-490c-ac88-b675e5e0defd None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:16.876140 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca98c8f-42df-490c-ac88-b675e5e0defd None admin] PATH_INFO: `/v3/projects/a7eea203d96a471cb590c418bb060d23/users/a6fda065d398415a9b8308457ab0f0dc/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:16.879128 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca98c8f-42df-490c-ac88-b675e5e0defd None admin] RBAC: Authorizing `identity:create_grant(project_id=a7eea203d96a471cb590c418bb060d23, user_id=a6fda065d398415a9b8308457ab0f0dc, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:16.880614 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca98c8f-42df-490c-ac88-b675e5e0defd None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:16.903101 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 65/126] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 14:12:16 2022] PUT /identity/v3/projects/a7eea203d96a471cb590c418bb060d23/users/a6fda065d398415a9b8308457ab0f0dc/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:12:16.904438 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca98c8f-42df-490c-ac88-b675e5e0defd None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:16.935852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2623e353-7ca1-48a1-a3eb-9b5848898253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:16.936890 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2623e353-7ca1-48a1-a3eb-9b5848898253 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:16.937376 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2623e353-7ca1-48a1-a3eb-9b5848898253 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:16.937851 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2623e353-7ca1-48a1-a3eb-9b5848898253 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:17.047200 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:17.075266 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2623e353-7ca1-48a1-a3eb-9b5848898253 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:17.076959 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2623e353-7ca1-48a1-a3eb-9b5848898253 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:17.080194 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:17.093325 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca98c8f-42df-490c-ac88-b675e5e0defd None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:17.132899 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec13df-ada4-4d82-81d3-dc8525943a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:17.133913 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec13df-ada4-4d82-81d3-dc8525943a24 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:17.134473 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec13df-ada4-4d82-81d3-dc8525943a24 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:17.135612 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec13df-ada4-4d82-81d3-dc8525943a24 None admin] PATH_INFO: `/v3/users/75510b26c21742e091c6eca468df1c06` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:17.137097 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec13df-ada4-4d82-81d3-dc8525943a24 None admin] RBAC: Authorizing `identity:get_user(user_id=75510b26c21742e091c6eca468df1c06)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:17.139007 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec13df-ada4-4d82-81d3-dc8525943a24 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:17.140951 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 66/127] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:12:16 2022] GET /identity/v3/users/75510b26c21742e091c6eca468df1c06 => generated 275 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:17.141965 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec13df-ada4-4d82-81d3-dc8525943a24 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:17.198918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 62/128] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:12:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:12:17.198918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2623e353-7ca1-48a1-a3eb-9b5848898253 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:17.278565 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec13df-ada4-4d82-81d3-dc8525943a24 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:17.348677 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:17.359853 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2623e353-7ca1-48a1-a3eb-9b5848898253 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:17.362986 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2284f49-0e48-4d73-9515-41a7982bedd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:17.363928 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2284f49-0e48-4d73-9515-41a7982bedd4 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:17.364404 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2284f49-0e48-4d73-9515-41a7982bedd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:17.365096 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2284f49-0e48-4d73-9515-41a7982bedd4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:17.368799 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:17.391520 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2284f49-0e48-4d73-9515-41a7982bedd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:17.404793 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-f2284f49-0e48-4d73-9515-41a7982bedd4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:12:17.404793 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:17.404793 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:17.404793 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:17.404793 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:17.404793 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:17.404793 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:17.404793 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:17.404793 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:17.404793 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:17.404793 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:17.404793 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:17.404793 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:17.404793 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:17.404793 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:17.404793 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:17.405941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:17.405941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:17.405941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:17.405941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:17.405941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:17.405941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:17.405941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:17.405941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:17.405941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:17.405941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:17.405941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:17.405941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:17.405941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:17.405941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:17.405941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:17.405941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:17.406685 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:17.406685 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:17.406685 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:17.406685 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:17.406685 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:17.406685 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:17.406685 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:17.406685 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:17.406685 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:17.406685 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:17.406685 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:12:17.406685 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:12:17.409083 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 67/129] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:12:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:12:17.410086 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2284f49-0e48-4d73-9515-41a7982bedd4 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:17.449919 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b15fa3-30c5-4c37-a250-8183f685b251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:17.451101 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b15fa3-30c5-4c37-a250-8183f685b251 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:17.451588 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b15fa3-30c5-4c37-a250-8183f685b251 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:17.452086 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b15fa3-30c5-4c37-a250-8183f685b251 None admin] PATH_INFO: `/v3/projects/a7eea203d96a471cb590c418bb060d23` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:17.455644 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b15fa3-30c5-4c37-a250-8183f685b251 None admin] RBAC: Authorizing `identity:get_project(project_id=a7eea203d96a471cb590c418bb060d23)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:17.455644 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b15fa3-30c5-4c37-a250-8183f685b251 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:17.457092 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 63/130] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:12:17 2022] GET /identity/v3/projects/a7eea203d96a471cb590c418bb060d23 => generated 298 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:17.458069 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b15fa3-30c5-4c37-a250-8183f685b251 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:17.641947 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b15fa3-30c5-4c37-a250-8183f685b251 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:17.672793 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2284f49-0e48-4d73-9515-41a7982bedd4 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:17.727324 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385a674-3842-49bc-9032-9c0a4ac77ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:17.727873 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385a674-3842-49bc-9032-9c0a4ac77ca3 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:17.728191 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385a674-3842-49bc-9032-9c0a4ac77ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:17.728473 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385a674-3842-49bc-9032-9c0a4ac77ca3 None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:17.729154 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a385a674-3842-49bc-9032-9c0a4ac77ca3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:17.729461 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a385a674-3842-49bc-9032-9c0a4ac77ca3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:17.730889 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a385a674-3842-49bc-9032-9c0a4ac77ca3 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:17.762451 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 64/131] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 14:12:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:17.763351 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0a6e4-6d47-469e-bad1-a8439f8f9fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:17.767964 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0a6e4-6d47-469e-bad1-a8439f8f9fd4 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:17.768077 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385a674-3842-49bc-9032-9c0a4ac77ca3 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:17.768151 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0a6e4-6d47-469e-bad1-a8439f8f9fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:17.768151 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0a6e4-6d47-469e-bad1-a8439f8f9fd4 None admin] PATH_INFO: `/v3/projects/a7eea203d96a471cb590c418bb060d23` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:17.771652 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0a6e4-6d47-469e-bad1-a8439f8f9fd4 None admin] RBAC: Authorizing `identity:get_project(project_id=a7eea203d96a471cb590c418bb060d23)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:17.777653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0a6e4-6d47-469e-bad1-a8439f8f9fd4 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:17.777777 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 68/132] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:12:17 2022] GET /identity/v3/projects/a7eea203d96a471cb590c418bb060d23 => generated 298 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:17.778600 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0a6e4-6d47-469e-bad1-a8439f8f9fd4 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:17.856877 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:17.931187 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:18.001775 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385a674-3842-49bc-9032-9c0a4ac77ca3 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:18.017732 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0a6e4-6d47-469e-bad1-a8439f8f9fd4 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:18.064881 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1b66cb-d1e2-48a8-b23a-48f41d31f642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:18.065615 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b66cb-d1e2-48a8-b23a-48f41d31f642 None admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:18.065904 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b66cb-d1e2-48a8-b23a-48f41d31f642 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:18.066183 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b66cb-d1e2-48a8-b23a-48f41d31f642 None admin] PATH_INFO: `/v3/projects/a7eea203d96a471cb590c418bb060d23/users/75510b26c21742e091c6eca468df1c06/roles/25257d00ad2640c2a57d257b0279155f` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:18.070515 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1b66cb-d1e2-48a8-b23a-48f41d31f642 None admin] RBAC: Authorizing `identity:create_grant(project_id=a7eea203d96a471cb590c418bb060d23, user_id=75510b26c21742e091c6eca468df1c06, role_id=25257d00ad2640c2a57d257b0279155f)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:18.072037 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1b66cb-d1e2-48a8-b23a-48f41d31f642 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:18.084389 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a769cc-556f-479e-b0ef-001eff089156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:18.085037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a769cc-556f-479e-b0ef-001eff089156 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:18.085385 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a769cc-556f-479e-b0ef-001eff089156 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:18.085618 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a769cc-556f-479e-b0ef-001eff089156 None admin] PATH_INFO: `/v3/users/75510b26c21742e091c6eca468df1c06` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:18.086978 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a769cc-556f-479e-b0ef-001eff089156 None admin] RBAC: Authorizing `identity:get_user(user_id=75510b26c21742e091c6eca468df1c06)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:18.089186 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a769cc-556f-479e-b0ef-001eff089156 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:18.091097 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 69/133] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:12:17 2022] GET /identity/v3/users/75510b26c21742e091c6eca468df1c06 => generated 275 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:18.091836 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a769cc-556f-479e-b0ef-001eff089156 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:18.100976 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 65/134] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 14:12:17 2022] PUT /identity/v3/projects/a7eea203d96a471cb590c418bb060d23/users/75510b26c21742e091c6eca468df1c06/roles/25257d00ad2640c2a57d257b0279155f => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:12:18.101879 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1b66cb-d1e2-48a8-b23a-48f41d31f642 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:18.314422 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a769cc-556f-479e-b0ef-001eff089156 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:18.316696 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:18.334270 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1b66cb-d1e2-48a8-b23a-48f41d31f642 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:18.349734 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:18.360653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6882fd2-d30b-4896-8667-8c753f5b0fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:18.361761 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6882fd2-d30b-4896-8667-8c753f5b0fe8 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:18.362474 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6882fd2-d30b-4896-8667-8c753f5b0fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:18.362968 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6882fd2-d30b-4896-8667-8c753f5b0fe8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:18.364449 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6882fd2-d30b-4896-8667-8c753f5b0fe8 None admin] RBAC: Adding query filter params (role.id=651d60b8bf604682a0609ba8ad8c3f1b, scope.project.id=a7eea203d96a471cb590c418bb060d23, user.id=44e73f0cd35648198c204a3f657edccb) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:18.365048 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6882fd2-d30b-4896-8667-8c753f5b0fe8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:18.366867 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6882fd2-d30b-4896-8667-8c753f5b0fe8 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:18.376660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 70/135] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:12:18 2022] GET /identity/v3/role_assignments?user.id=44e73f0cd35648198c204a3f657edccb&scope.project.id=a7eea203d96a471cb590c418bb060d23&role.id=651d60b8bf604682a0609ba8ad8c3f1b => generated 259 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:18.410075 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d5460e-9357-44d6-af0c-f1d72ae9ccdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:18.410933 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5460e-9357-44d6-af0c-f1d72ae9ccdf None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:18.411344 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5460e-9357-44d6-af0c-f1d72ae9ccdf None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:18.411726 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5460e-9357-44d6-af0c-f1d72ae9ccdf None admin] PATH_INFO: `/v3/projects/f7af21ee123e483ea1be1cb0e7f9f73b` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:18.415749 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d5460e-9357-44d6-af0c-f1d72ae9ccdf None admin] RBAC: Authorizing `identity:get_project(project_id=f7af21ee123e483ea1be1cb0e7f9f73b)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:18.417372 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d5460e-9357-44d6-af0c-f1d72ae9ccdf None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:18.419245 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 66/136] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:12:18 2022] GET /identity/v3/projects/f7af21ee123e483ea1be1cb0e7f9f73b => generated 312 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:18.425441 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6882fd2-d30b-4896-8667-8c753f5b0fe8 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:18.562462 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6882fd2-d30b-4896-8667-8c753f5b0fe8 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:18.607054 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeb8641-92ec-4c7f-b287-e72906b90c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:18.607996 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb8641-92ec-4c7f-b287-e72906b90c33 None admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:18.608468 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb8641-92ec-4c7f-b287-e72906b90c33 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:18.608880 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb8641-92ec-4c7f-b287-e72906b90c33 None admin] PATH_INFO: `/v3/projects/f7af21ee123e483ea1be1cb0e7f9f73b/users/75510b26c21742e091c6eca468df1c06/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:18.611266 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeb8641-92ec-4c7f-b287-e72906b90c33 None admin] RBAC: Authorizing `identity:create_grant(project_id=f7af21ee123e483ea1be1cb0e7f9f73b, user_id=75510b26c21742e091c6eca468df1c06, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:18.612950 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeb8641-92ec-4c7f-b287-e72906b90c33 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:18.629070 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 71/137] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 14:12:18 2022] PUT /identity/v3/projects/f7af21ee123e483ea1be1cb0e7f9f73b/users/75510b26c21742e091c6eca468df1c06/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:12:18.716651 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fefa0-f6e3-4a32-84c8-08ab9382534e None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:18.716939 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fefa0-f6e3-4a32-84c8-08ab9382534e None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:18.717220 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fefa0-f6e3-4a32-84c8-08ab9382534e None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:18.718618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 67/138] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:18.724592 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e35a8c-c1cc-4cfd-ae57-093462cc37d4 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:18.725085 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e35a8c-c1cc-4cfd-ae57-093462cc37d4 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:18.725299 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e35a8c-c1cc-4cfd-ae57-093462cc37d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:18.752237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-c3e35a8c-c1cc-4cfd-ae57-093462cc37d4 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:18.753530 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-c3e35a8c-c1cc-4cfd-ae57-093462cc37d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:18.811930 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa99850-f22a-4f24-897e-167940bab3d7 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:18.812542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa99850-f22a-4f24-897e-167940bab3d7 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:18.813018 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa99850-f22a-4f24-897e-167940bab3d7 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:18.815584 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 68/139] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:18.820068 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:18.821652 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344d5aa8-919c-452d-b6b4-a7cf474946ad None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:18.822221 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344d5aa8-919c-452d-b6b4-a7cf474946ad None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:18.822703 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344d5aa8-919c-452d-b6b4-a7cf474946ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:18.848182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 72/140] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:18.876155 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-344d5aa8-919c-452d-b6b4-a7cf474946ad None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:18.877709 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-344d5aa8-919c-452d-b6b4-a7cf474946ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:18.958546 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 69/141] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:19.202387 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e35a8c-c1cc-4cfd-ae57-093462cc37d4 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:19.332654 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e35a8c-c1cc-4cfd-ae57-093462cc37d4 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:19.381366 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e47bebc-8478-4d1f-9ed1-c6e9a6aadf01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:19.382012 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47bebc-8478-4d1f-9ed1-c6e9a6aadf01 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:19.382293 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47bebc-8478-4d1f-9ed1-c6e9a6aadf01 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:19.382564 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47bebc-8478-4d1f-9ed1-c6e9a6aadf01 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:19.383711 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e47bebc-8478-4d1f-9ed1-c6e9a6aadf01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:19.385735 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e47bebc-8478-4d1f-9ed1-c6e9a6aadf01 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:19.387521 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 73/142] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:12:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:19.392921 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344d5aa8-919c-452d-b6b4-a7cf474946ad None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:19.473789 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e47bebc-8478-4d1f-9ed1-c6e9a6aadf01 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:19.500364 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:19.516801 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344d5aa8-919c-452d-b6b4-a7cf474946ad None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:19.555240 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:19.582328 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a4fbf-3ea0-455e-9afe-7ad0d18e1f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:19.583205 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a4fbf-3ea0-455e-9afe-7ad0d18e1f47 None admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:19.583781 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a4fbf-3ea0-455e-9afe-7ad0d18e1f47 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:19.585336 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a4fbf-3ea0-455e-9afe-7ad0d18e1f47 None admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:19.586139 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476a4fbf-3ea0-455e-9afe-7ad0d18e1f47 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:19.592762 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476a4fbf-3ea0-455e-9afe-7ad0d18e1f47 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:19.621080 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 70/143] 10.222.0.52 () {60 vars in 1132 bytes} [Sat Sep 24 14:12:19 2022] POST /identity/v3/users => generated 283 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:12:19.641800 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e47bebc-8478-4d1f-9ed1-c6e9a6aadf01 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:19.699673 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41230b07-ebcf-4431-8f84-9a4634a2a732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:19.700795 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41230b07-ebcf-4431-8f84-9a4634a2a732 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:19.701266 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41230b07-ebcf-4431-8f84-9a4634a2a732 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:19.701750 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41230b07-ebcf-4431-8f84-9a4634a2a732 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:19.713830 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41230b07-ebcf-4431-8f84-9a4634a2a732 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:19.719158 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-41230b07-ebcf-4431-8f84-9a4634a2a732 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:12:19.719158 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:19.719158 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:19.719158 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:19.719158 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:19.719158 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:19.719158 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:19.719158 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:19.719158 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:19.719158 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:19.719158 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:19.719158 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:19.719158 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:19.719158 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:19.719158 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:19.719158 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:19.720000 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:19.720000 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:19.720000 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:19.720000 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:19.720000 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:19.720000 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:19.720000 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:19.720000 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:19.720000 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:19.720000 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:19.720000 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:19.720000 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:19.720000 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:19.720000 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:19.720000 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:19.720000 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:19.720773 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:19.720773 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:19.720773 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:19.720773 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:19.720773 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:19.720773 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:19.720773 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:19.720773 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:19.720773 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:19.720773 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:19.720773 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:12:19.720773 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:12:19.723203 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 74/144] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:12:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:12:19.729872 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a4fbf-3ea0-455e-9afe-7ad0d18e1f47 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:19.898031 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a4fbf-3ea0-455e-9afe-7ad0d18e1f47 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:19.952934 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522110c9-c013-49b9-b799-fe541901742c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:19.953558 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522110c9-c013-49b9-b799-fe541901742c None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:19.953843 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522110c9-c013-49b9-b799-fe541901742c None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:19.954122 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522110c9-c013-49b9-b799-fe541901742c None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:19.954806 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522110c9-c013-49b9-b799-fe541901742c None admin] RBAC: Adding query filter params (name=member) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:19.955146 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522110c9-c013-49b9-b799-fe541901742c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:19.956537 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522110c9-c013-49b9-b799-fe541901742c None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:19.970708 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 71/145] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 14:12:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:19.976598 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41230b07-ebcf-4431-8f84-9a4634a2a732 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:20.136481 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41230b07-ebcf-4431-8f84-9a4634a2a732 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:20.147590 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:20.188451 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e0a39-1bfb-4eee-a0f0-e78a2110b8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:20.189102 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e0a39-1bfb-4eee-a0f0-e78a2110b8b5 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:20.189386 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e0a39-1bfb-4eee-a0f0-e78a2110b8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:20.189665 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e0a39-1bfb-4eee-a0f0-e78a2110b8b5 None admin] PATH_INFO: `/v3/users/75510b26c21742e091c6eca468df1c06` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:20.190832 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e0a39-1bfb-4eee-a0f0-e78a2110b8b5 None admin] RBAC: Authorizing `identity:get_user(user_id=75510b26c21742e091c6eca468df1c06)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:20.192305 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e0a39-1bfb-4eee-a0f0-e78a2110b8b5 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:20.193898 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 75/146] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:12:19 2022] GET /identity/v3/users/75510b26c21742e091c6eca468df1c06 => generated 275 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:20.200139 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522110c9-c013-49b9-b799-fe541901742c None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:20.209818 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aba3c3-5211-4d75-9bc9-479972ce0a52 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:20.210108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aba3c3-5211-4d75-9bc9-479972ce0a52 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:20.210402 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aba3c3-5211-4d75-9bc9-479972ce0a52 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:20.211765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 76/147] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:20.218157 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:20.218782 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfad64-9677-4a41-83c1-884e491c6346 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:20.219324 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfad64-9677-4a41-83c1-884e491c6346 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:20.219789 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfad64-9677-4a41-83c1-884e491c6346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:20.244968 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-6adfad64-9677-4a41-83c1-884e491c6346 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:20.246494 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-6adfad64-9677-4a41-83c1-884e491c6346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:20.319834 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 77/148] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:12:20.385965 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522110c9-c013-49b9-b799-fe541901742c None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:20.439771 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416fb262-c8ed-461e-be97-ea107af82603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:20.440328 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416fb262-c8ed-461e-be97-ea107af82603 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:20.440613 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416fb262-c8ed-461e-be97-ea107af82603 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:20.440892 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416fb262-c8ed-461e-be97-ea107af82603 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:20.523328 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416fb262-c8ed-461e-be97-ea107af82603 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:20.524871 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416fb262-c8ed-461e-be97-ea107af82603 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:20.540901 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:20.635750 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adfad64-9677-4a41-83c1-884e491c6346 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:20.635970 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 72/149] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:12:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:12:20.641380 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416fb262-c8ed-461e-be97-ea107af82603 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:20.776372 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416fb262-c8ed-461e-be97-ea107af82603 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:20.796356 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:20.838844 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0397c-8f4e-4ff5-874e-f23ce28beaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:20.838844 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0397c-8f4e-4ff5-874e-f23ce28beaab None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:20.838844 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0397c-8f4e-4ff5-874e-f23ce28beaab None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:20.838844 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0397c-8f4e-4ff5-874e-f23ce28beaab None admin] PATH_INFO: `/v3/projects/a7eea203d96a471cb590c418bb060d23` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:20.840073 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0397c-8f4e-4ff5-874e-f23ce28beaab None admin] RBAC: Authorizing `identity:get_project(project_id=a7eea203d96a471cb590c418bb060d23)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:20.841966 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0397c-8f4e-4ff5-874e-f23ce28beaab None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:20.843720 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 73/150] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:12:20 2022] GET /identity/v3/projects/a7eea203d96a471cb590c418bb060d23 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:20.847588 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adfad64-9677-4a41-83c1-884e491c6346 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:20.849829 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0397c-8f4e-4ff5-874e-f23ce28beaab None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:20.869707 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:20.918080 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604b890-fb57-44c1-b21b-754d18078bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:20.918792 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604b890-fb57-44c1-b21b-754d18078bdd None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:20.919118 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604b890-fb57-44c1-b21b-754d18078bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:20.919388 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604b890-fb57-44c1-b21b-754d18078bdd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:20.937371 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d604b890-fb57-44c1-b21b-754d18078bdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:20.942953 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-d604b890-fb57-44c1-b21b-754d18078bdd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:12:20.942953 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:20.942953 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:20.942953 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:20.942953 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:20.942953 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:20.942953 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:20.942953 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:20.942953 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:20.942953 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:20.942953 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:20.942953 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:20.942953 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:20.942953 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:20.942953 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:20.942953 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:20.943865 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:20.943865 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:20.943865 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:20.943865 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:20.943865 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:20.943865 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:20.943865 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:20.943865 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:20.943865 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:20.943865 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:20.943865 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:20.943865 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:20.943865 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:20.943865 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:20.943865 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:20.943865 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:20.944975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:20.944975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:20.944975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:20.944975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:20.944975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:20.944975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:20.944975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:20.944975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:20.944975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:20.944975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:20.944975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:12:20.944975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:12:20.944975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 78/151] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 14:12:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:12:20.956880 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604b890-fb57-44c1-b21b-754d18078bdd None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:21.025772 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0397c-8f4e-4ff5-874e-f23ce28beaab None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:21.126920 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72014cc9-403b-4644-80de-b99968224d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:21.126920 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72014cc9-403b-4644-80de-b99968224d93 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:21.126920 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72014cc9-403b-4644-80de-b99968224d93 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:21.126920 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72014cc9-403b-4644-80de-b99968224d93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:21.127290 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72014cc9-403b-4644-80de-b99968224d93 None admin] RBAC: Adding query filter params (role.id=640af5182e6848498c16a19cddbfb738, scope.project.id=a7eea203d96a471cb590c418bb060d23, user.id=75510b26c21742e091c6eca468df1c06) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:21.127290 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72014cc9-403b-4644-80de-b99968224d93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:21.128501 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72014cc9-403b-4644-80de-b99968224d93 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:21.140898 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604b890-fb57-44c1-b21b-754d18078bdd None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:21.142850 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 74/152] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:12:20 2022] GET /identity/v3/role_assignments?user.id=75510b26c21742e091c6eca468df1c06&scope.project.id=a7eea203d96a471cb590c418bb060d23&role.id=640af5182e6848498c16a19cddbfb738 => generated 608 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:21.192491 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214c4561-e68a-4721-a17f-37494604ab8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:21.193481 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c4561-e68a-4721-a17f-37494604ab8f None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:21.193964 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c4561-e68a-4721-a17f-37494604ab8f None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:21.194438 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c4561-e68a-4721-a17f-37494604ab8f None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:21.195348 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214c4561-e68a-4721-a17f-37494604ab8f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:21.195860 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214c4561-e68a-4721-a17f-37494604ab8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:21.198044 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214c4561-e68a-4721-a17f-37494604ab8f None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:21.211250 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 79/153] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:12:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:21.215223 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72014cc9-403b-4644-80de-b99968224d93 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:21.285180 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:21.359630 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72014cc9-403b-4644-80de-b99968224d93 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:21.403057 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba4721-e9e5-4f19-93e4-f7bf5b200e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:21.403714 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba4721-e9e5-4f19-93e4-f7bf5b200e8a None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:21.403996 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba4721-e9e5-4f19-93e4-f7bf5b200e8a None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:21.404271 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba4721-e9e5-4f19-93e4-f7bf5b200e8a None admin] PATH_INFO: `/v3/users/a6fda065d398415a9b8308457ab0f0dc` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:21.405718 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ba4721-e9e5-4f19-93e4-f7bf5b200e8a None admin] RBAC: Authorizing `identity:get_user(user_id=a6fda065d398415a9b8308457ab0f0dc)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:21.407247 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ba4721-e9e5-4f19-93e4-f7bf5b200e8a None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:21.409187 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 75/154] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:12:21 2022] GET /identity/v3/users/a6fda065d398415a9b8308457ab0f0dc => generated 247 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:21.414504 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214c4561-e68a-4721-a17f-37494604ab8f None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:21.451920 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:21.535034 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214c4561-e68a-4721-a17f-37494604ab8f None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:21.572635 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff8563-19b0-4ccb-99a0-4d3eab72431f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:21.573263 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff8563-19b0-4ccb-99a0-4d3eab72431f None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:21.573536 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff8563-19b0-4ccb-99a0-4d3eab72431f None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:21.573799 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff8563-19b0-4ccb-99a0-4d3eab72431f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:21.654457 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ff8563-19b0-4ccb-99a0-4d3eab72431f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:21.656140 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ff8563-19b0-4ccb-99a0-4d3eab72431f None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:21.663459 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851e8fa-4efb-4874-aad7-0bf79263442b None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:21.663934 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851e8fa-4efb-4874-aad7-0bf79263442b None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:21.664024 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851e8fa-4efb-4874-aad7-0bf79263442b None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:21.665519 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 76/155] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:12:21.670651 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3251c-12e3-4853-84e6-9e2ca4cd1497 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:21.670963 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3251c-12e3-4853-84e6-9e2ca4cd1497 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:21.671243 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3251c-12e3-4853-84e6-9e2ca4cd1497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:21.692284 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:21.718147 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-9bf3251c-12e3-4853-84e6-9e2ca4cd1497 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:21.724251 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-9bf3251c-12e3-4853-84e6-9e2ca4cd1497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:21.792316 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 80/156] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:12:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:12:21.797865 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff8563-19b0-4ccb-99a0-4d3eab72431f None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:21.856480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 77/157] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:21.918438 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff8563-19b0-4ccb-99a0-4d3eab72431f None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:21.957158 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:21.966954 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e57b4-ae76-4a55-ba06-d0162bccc9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:21.967921 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e57b4-ae76-4a55-ba06-d0162bccc9a8 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:21.968396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e57b4-ae76-4a55-ba06-d0162bccc9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:21.968841 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e57b4-ae76-4a55-ba06-d0162bccc9a8 None admin] PATH_INFO: `/v3/projects/a7eea203d96a471cb590c418bb060d23` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:21.974592 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869e57b4-ae76-4a55-ba06-d0162bccc9a8 None admin] RBAC: Authorizing `identity:get_project(project_id=a7eea203d96a471cb590c418bb060d23)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:21.976423 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869e57b4-ae76-4a55-ba06-d0162bccc9a8 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:21.978594 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 81/158] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:12:21 2022] GET /identity/v3/projects/a7eea203d96a471cb590c418bb060d23 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:21.984747 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf3251c-12e3-4853-84e6-9e2ca4cd1497 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:22.040933 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:22.090771 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345f935-3f12-4413-b5d7-0da67544cfd0 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:22.091169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345f935-3f12-4413-b5d7-0da67544cfd0 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:22.091514 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345f935-3f12-4413-b5d7-0da67544cfd0 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:22.094499 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 82/159] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:22.098741 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5413be-c494-43ed-b863-a9f7a39fd024 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:22.099167 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5413be-c494-43ed-b863-a9f7a39fd024 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:22.099515 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5413be-c494-43ed-b863-a9f7a39fd024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:22.111224 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf3251c-12e3-4853-84e6-9e2ca4cd1497 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:22.136121 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-3c5413be-c494-43ed-b863-a9f7a39fd024 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:22.137332 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-3c5413be-c494-43ed-b863-a9f7a39fd024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:22.146526 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bf5b16-74e0-4374-b9cc-e0fd5b83174c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:22.147170 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf5b16-74e0-4374-b9cc-e0fd5b83174c None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:22.147447 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf5b16-74e0-4374-b9cc-e0fd5b83174c None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:22.147718 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf5b16-74e0-4374-b9cc-e0fd5b83174c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:22.148448 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bf5b16-74e0-4374-b9cc-e0fd5b83174c None admin] RBAC: Adding query filter params (role.id=d00b78c32dbf4cc7925a55bb621d8fcf, scope.project.id=a7eea203d96a471cb590c418bb060d23, user.id=a6fda065d398415a9b8308457ab0f0dc) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:22.148749 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bf5b16-74e0-4374-b9cc-e0fd5b83174c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:22.150836 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bf5b16-74e0-4374-b9cc-e0fd5b83174c None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:22.157373 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 78/160] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:12:21 2022] GET /identity/v3/role_assignments?user.id=a6fda065d398415a9b8308457ab0f0dc&scope.project.id=a7eea203d96a471cb590c418bb060d23&role.id=d00b78c32dbf4cc7925a55bb621d8fcf => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:22.257482 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 83/161] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:12:22.396882 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70115447-c21a-4e49-886c-cbc78e144fd9 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:22.397184 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70115447-c21a-4e49-886c-cbc78e144fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:22.397426 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70115447-c21a-4e49-886c-cbc78e144fd9 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:22.398933 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 79/162] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:22.402026 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5413be-c494-43ed-b863-a9f7a39fd024 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:22.404248 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569c3b1-5a09-46aa-9c1b-7f54a15183c9 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:22.404534 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569c3b1-5a09-46aa-9c1b-7f54a15183c9 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:22.404807 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569c3b1-5a09-46aa-9c1b-7f54a15183c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:22.425548 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-4569c3b1-5a09-46aa-9c1b-7f54a15183c9 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:22.426738 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-4569c3b1-5a09-46aa-9c1b-7f54a15183c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:22.505941 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 80/163] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:22.555112 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5413be-c494-43ed-b863-a9f7a39fd024 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:22.593165 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:22.602879 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b355d-1443-475d-aaad-30ee3cb3b5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:22.603722 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b355d-1443-475d-aaad-30ee3cb3b5be None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:22.604001 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b355d-1443-475d-aaad-30ee3cb3b5be None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:22.604316 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b355d-1443-475d-aaad-30ee3cb3b5be None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:22.615043 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99b355d-1443-475d-aaad-30ee3cb3b5be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:22.620415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-f99b355d-1443-475d-aaad-30ee3cb3b5be None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:12:22.620415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:22.620415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:22.620415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:22.620415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:22.620415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:22.620415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:22.620415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:22.620415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:22.620415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:22.620415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:22.620415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:22.620415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:22.620415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:22.620415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:22.620415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:22.621237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:22.621237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:22.621237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:22.621237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:22.621237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:22.621237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:22.621237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:22.621237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:22.621237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:22.621237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:22.621237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:22.621237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:22.621237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:22.621237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:22.621237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:22.621237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:22.622059 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:22.622059 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:22.622059 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:22.622059 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:22.622059 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:22.622059 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:22.622059 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:22.622059 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:22.622059 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:22.622059 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:22.622059 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:12:22.622059 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:12:22.622059 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 84/164] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 14:12:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:12:22.628577 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4569c3b1-5a09-46aa-9c1b-7f54a15183c9 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:22.678690 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:22.747230 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b355d-1443-475d-aaad-30ee3cb3b5be None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:22.771880 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4569c3b1-5a09-46aa-9c1b-7f54a15183c9 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:22.810993 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0114ae-cfc0-4e85-a264-3345a8b62b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:22.811533 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0114ae-cfc0-4e85-a264-3345a8b62b76 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:22.812090 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0114ae-cfc0-4e85-a264-3345a8b62b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:22.812362 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0114ae-cfc0-4e85-a264-3345a8b62b76 None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:22.813038 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0114ae-cfc0-4e85-a264-3345a8b62b76 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:22.813339 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0114ae-cfc0-4e85-a264-3345a8b62b76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:22.814783 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0114ae-cfc0-4e85-a264-3345a8b62b76 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:22.823524 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 81/165] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:12:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:22.830089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0114ae-cfc0-4e85-a264-3345a8b62b76 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:22.939216 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:22.942271 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0114ae-cfc0-4e85-a264-3345a8b62b76 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:22.970763 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b355d-1443-475d-aaad-30ee3cb3b5be None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:22.979329 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5bb475-addd-4b90-9171-8f706a29dccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:22.980337 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5bb475-addd-4b90-9171-8f706a29dccb None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:22.980467 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5bb475-addd-4b90-9171-8f706a29dccb None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:22.980728 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5bb475-addd-4b90-9171-8f706a29dccb None admin] PATH_INFO: `/v3/users/75510b26c21742e091c6eca468df1c06` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:22.982112 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5bb475-addd-4b90-9171-8f706a29dccb None admin] RBAC: Authorizing `identity:get_user(user_id=75510b26c21742e091c6eca468df1c06)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:22.983762 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5bb475-addd-4b90-9171-8f706a29dccb None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:22.985697 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 82/166] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:12:22 2022] GET /identity/v3/users/75510b26c21742e091c6eca468df1c06 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:22.987457 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5bb475-addd-4b90-9171-8f706a29dccb None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:23.023314 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254ef0eb-f355-4049-86fa-a6d0a3ba42e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:23.024453 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ef0eb-f355-4049-86fa-a6d0a3ba42e1 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:23.024803 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ef0eb-f355-4049-86fa-a6d0a3ba42e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:23.025899 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ef0eb-f355-4049-86fa-a6d0a3ba42e1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:23.041582 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254ef0eb-f355-4049-86fa-a6d0a3ba42e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:23.048537 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-254ef0eb-f355-4049-86fa-a6d0a3ba42e1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:12:23.048537 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:23.048537 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:23.048537 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:23.048537 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:23.048537 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:23.048537 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:23.048537 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:23.048537 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:23.048537 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:23.048537 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:23.048537 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:23.048537 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:23.048537 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:23.048537 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:23.048537 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:23.049433 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:23.049433 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:23.049433 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:23.049433 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:23.049433 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:23.049433 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:23.049433 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:23.049433 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:23.049433 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:23.049433 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:23.049433 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:23.049433 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:23.049433 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:23.049433 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:23.049433 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:23.049433 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:23.050662 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:23.050662 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:23.050662 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:23.050662 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:23.050662 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:23.050662 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:23.050662 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:23.050662 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:23.050662 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:23.050662 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:23.050662 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:12:23.050662 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:12:23.050662 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 85/167] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:12:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:12:23.050662 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254ef0eb-f355-4049-86fa-a6d0a3ba42e1 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:23.135170 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5bb475-addd-4b90-9171-8f706a29dccb None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:23.136796 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:23.193637 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f2149-cba1-4d0c-9591-332e6fc5993d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:23.194298 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f2149-cba1-4d0c-9591-332e6fc5993d None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:23.194578 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f2149-cba1-4d0c-9591-332e6fc5993d None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:23.194874 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f2149-cba1-4d0c-9591-332e6fc5993d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:23.206685 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f2149-cba1-4d0c-9591-332e6fc5993d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:23.213110 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-8b5f2149-cba1-4d0c-9591-332e6fc5993d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 14:12:23.213110 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:23.213110 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:23.213110 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:23.213110 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:23.213110 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:23.213110 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:23.213110 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:23.213110 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:23.213110 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:23.213110 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:23.213110 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:23.213110 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:23.213110 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:23.213110 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:23.213110 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:23.213967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:23.213967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:23.213967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:23.213967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:23.213967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:23.213967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:23.213967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:23.213967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:23.213967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:23.213967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:23.213967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:23.213967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:23.213967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:23.213967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:23.213967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:23.213967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:23.215142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:23.215142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:23.215142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:23.215142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:23.215142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:23.215142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:23.215142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:23.215142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:23.215142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:23.215142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:23.215142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 14:12:23.215142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:12:23.215142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 83/168] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:12:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:12:23.215896 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f2149-cba1-4d0c-9591-332e6fc5993d None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:23.244671 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254ef0eb-f355-4049-86fa-a6d0a3ba42e1 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:23.266451 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:23.317350 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f3d9c-d8dd-4545-9454-8d5ff40ba712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:23.318656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f3d9c-d8dd-4545-9454-8d5ff40ba712 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:23.319610 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f3d9c-d8dd-4545-9454-8d5ff40ba712 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:23.320081 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f3d9c-d8dd-4545-9454-8d5ff40ba712 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:23.382703 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f2149-cba1-4d0c-9591-332e6fc5993d None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:23.430381 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17c3cd-b99b-4158-a073-44778cf554d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:23.430950 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17c3cd-b99b-4158-a073-44778cf554d1 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:23.431230 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17c3cd-b99b-4158-a073-44778cf554d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:23.431542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17c3cd-b99b-4158-a073-44778cf554d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:23.432217 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed17c3cd-b99b-4158-a073-44778cf554d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:23.432517 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed17c3cd-b99b-4158-a073-44778cf554d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:23.433922 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed17c3cd-b99b-4158-a073-44778cf554d1 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:23.438556 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386f3d9c-d8dd-4545-9454-8d5ff40ba712 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:23.440142 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386f3d9c-d8dd-4545-9454-8d5ff40ba712 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:23.455418 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 84/169] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 14:12:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:23.456254 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17c3cd-b99b-4158-a073-44778cf554d1 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:23.525741 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:23.581087 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 86/170] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:12:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:12:23.582229 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f3d9c-d8dd-4545-9454-8d5ff40ba712 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:23.651184 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:23.663612 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17c3cd-b99b-4158-a073-44778cf554d1 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:23.702237 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f8388-0df8-4c85-8df8-dffcb9a2c8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:23.704600 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f8388-0df8-4c85-8df8-dffcb9a2c8b0 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:23.705105 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f8388-0df8-4c85-8df8-dffcb9a2c8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:23.705555 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f8388-0df8-4c85-8df8-dffcb9a2c8b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:23.706417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80f8388-0df8-4c85-8df8-dffcb9a2c8b0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:23.706974 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80f8388-0df8-4c85-8df8-dffcb9a2c8b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:23.708535 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80f8388-0df8-4c85-8df8-dffcb9a2c8b0 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:23.719090 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 85/171] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 14:12:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:23.720322 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f8388-0df8-4c85-8df8-dffcb9a2c8b0 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:23.781202 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f3d9c-d8dd-4545-9454-8d5ff40ba712 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:23.854624 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a5da7-5de0-44d3-bd3a-6ada4c865c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:23.856362 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a5da7-5de0-44d3-bd3a-6ada4c865c0d None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:23.857437 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a5da7-5de0-44d3-bd3a-6ada4c865c0d None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:23.858056 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a5da7-5de0-44d3-bd3a-6ada4c865c0d None admin] PATH_INFO: `/v3/users/75510b26c21742e091c6eca468df1c06` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:23.860422 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4a5da7-5de0-44d3-bd3a-6ada4c865c0d None admin] RBAC: Authorizing `identity:get_user(user_id=75510b26c21742e091c6eca468df1c06)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:23.862221 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4a5da7-5de0-44d3-bd3a-6ada4c865c0d None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:23.865463 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 87/172] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:12:23 2022] GET /identity/v3/users/75510b26c21742e091c6eca468df1c06 => generated 275 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:23.866571 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a5da7-5de0-44d3-bd3a-6ada4c865c0d None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:23.874252 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f8388-0df8-4c85-8df8-dffcb9a2c8b0 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:23.904106 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:23.950781 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dafd9e-0e95-471f-85c1-b720b6b959a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:23.951777 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dafd9e-0e95-471f-85c1-b720b6b959a5 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:23.952310 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dafd9e-0e95-471f-85c1-b720b6b959a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:23.952760 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dafd9e-0e95-471f-85c1-b720b6b959a5 None admin] PATH_INFO: `/v3/projects/a7eea203d96a471cb590c418bb060d23` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:23.954513 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dafd9e-0e95-471f-85c1-b720b6b959a5 None admin] RBAC: Authorizing `identity:get_project(project_id=a7eea203d96a471cb590c418bb060d23)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:23.956237 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dafd9e-0e95-471f-85c1-b720b6b959a5 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:23.961891 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 86/173] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:12:23 2022] GET /identity/v3/projects/a7eea203d96a471cb590c418bb060d23 => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:23.962932 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dafd9e-0e95-471f-85c1-b720b6b959a5 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:24.004104 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:24.051516 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a5da7-5de0-44d3-bd3a-6ada4c865c0d None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:24.093246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae95a02-897b-45bf-856d-0a7681c3a1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:24.094293 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae95a02-897b-45bf-856d-0a7681c3a1d8 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:24.094805 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae95a02-897b-45bf-856d-0a7681c3a1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:24.095381 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae95a02-897b-45bf-856d-0a7681c3a1d8 None admin] PATH_INFO: `/v3/users/44e73f0cd35648198c204a3f657edccb` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:24.096730 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae95a02-897b-45bf-856d-0a7681c3a1d8 None admin] RBAC: Authorizing `identity:get_user(user_id=44e73f0cd35648198c204a3f657edccb)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:24.098379 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae95a02-897b-45bf-856d-0a7681c3a1d8 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:24.100262 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 88/174] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:12:23 2022] GET /identity/v3/users/44e73f0cd35648198c204a3f657edccb => generated 289 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:24.101297 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae95a02-897b-45bf-856d-0a7681c3a1d8 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:24.147538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dafd9e-0e95-471f-85c1-b720b6b959a5 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:24.211937 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c3e831-b256-4a5b-86f7-4fdc71524d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:24.213921 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3e831-b256-4a5b-86f7-4fdc71524d16 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:24.215168 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3e831-b256-4a5b-86f7-4fdc71524d16 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:24.216238 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3e831-b256-4a5b-86f7-4fdc71524d16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:24.285402 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:24.303776 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae95a02-897b-45bf-856d-0a7681c3a1d8 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:24.327818 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c3e831-b256-4a5b-86f7-4fdc71524d16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:24.329538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c3e831-b256-4a5b-86f7-4fdc71524d16 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:24.347785 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:24.352333 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9390302-f3e4-4fcb-a91c-59101827033e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:24.353075 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9390302-f3e4-4fcb-a91c-59101827033e None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:24.353968 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9390302-f3e4-4fcb-a91c-59101827033e None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:24.354372 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9390302-f3e4-4fcb-a91c-59101827033e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:24.356106 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9390302-f3e4-4fcb-a91c-59101827033e None admin] RBAC: Adding query filter params (role.id=25257d00ad2640c2a57d257b0279155f, scope.project.id=a7eea203d96a471cb590c418bb060d23, user.id=75510b26c21742e091c6eca468df1c06) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:24.356517 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9390302-f3e4-4fcb-a91c-59101827033e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:24.358847 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9390302-f3e4-4fcb-a91c-59101827033e None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:24.374867 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 89/175] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:12:24 2022] GET /identity/v3/role_assignments?user.id=75510b26c21742e091c6eca468df1c06&scope.project.id=a7eea203d96a471cb590c418bb060d23&role.id=25257d00ad2640c2a57d257b0279155f => generated 608 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:24.374867 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9390302-f3e4-4fcb-a91c-59101827033e None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:24.475429 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 87/176] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:12:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:12:24.477253 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad28402-0039-4d72-a690-15300c3f848c None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:24.477709 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad28402-0039-4d72-a690-15300c3f848c None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:24.478133 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad28402-0039-4d72-a690-15300c3f848c None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:24.480845 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 88/177] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:24.482312 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4dc86e-3175-459d-a161-eb7ad81a77ee None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:24.482773 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4dc86e-3175-459d-a161-eb7ad81a77ee None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:24.483242 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4dc86e-3175-459d-a161-eb7ad81a77ee None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:24.484624 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 89/178] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:24.490128 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4dc86e-3175-459d-a161-eb7ad81a77ee None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:24.511580 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9390302-f3e4-4fcb-a91c-59101827033e None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:24.549131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5c2be-c91b-4983-9b91-8a45d4be9b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:24.549780 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5c2be-c91b-4983-9b91-8a45d4be9b1c None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:24.550063 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5c2be-c91b-4983-9b91-8a45d4be9b1c None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:24.550341 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5c2be-c91b-4983-9b91-8a45d4be9b1c None admin] PATH_INFO: `/v3/projects/a7eea203d96a471cb590c418bb060d23` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:24.551521 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c5c2be-c91b-4983-9b91-8a45d4be9b1c None admin] RBAC: Authorizing `identity:get_project(project_id=a7eea203d96a471cb590c418bb060d23)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:24.552988 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c5c2be-c91b-4983-9b91-8a45d4be9b1c None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:24.554636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 90/179] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:12:24 2022] GET /identity/v3/projects/a7eea203d96a471cb590c418bb060d23 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:24.555791 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c91de-c37a-4002-9f8c-fe6015c63d90 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:24.556076 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c91de-c37a-4002-9f8c-fe6015c63d90 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:24.556404 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c91de-c37a-4002-9f8c-fe6015c63d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:24.576849 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-873c91de-c37a-4002-9f8c-fe6015c63d90 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:24.578053 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-873c91de-c37a-4002-9f8c-fe6015c63d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:24.591415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:24.619409 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4dc86e-3175-459d-a161-eb7ad81a77ee None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:24.638431 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:24.654245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 91/180] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:24.655787 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25285cfe-de6f-43fd-a179-0a8f957aaf2c None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:24.656130 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25285cfe-de6f-43fd-a179-0a8f957aaf2c None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:24.656359 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25285cfe-de6f-43fd-a179-0a8f957aaf2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:24.657909 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778bf2e0-9a76-49bb-982c-bb088233a87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:24.658541 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778bf2e0-9a76-49bb-982c-bb088233a87a None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:24.658840 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778bf2e0-9a76-49bb-982c-bb088233a87a None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:24.659122 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778bf2e0-9a76-49bb-982c-bb088233a87a None admin] PATH_INFO: `/v3/projects/f7af21ee123e483ea1be1cb0e7f9f73b` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:24.660199 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778bf2e0-9a76-49bb-982c-bb088233a87a None admin] RBAC: Authorizing `identity:get_project(project_id=f7af21ee123e483ea1be1cb0e7f9f73b)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:24.661632 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778bf2e0-9a76-49bb-982c-bb088233a87a None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:24.664252 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 90/181] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:12:24 2022] GET /identity/v3/projects/f7af21ee123e483ea1be1cb0e7f9f73b => generated 312 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:24.665378 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9886c-2d10-4d71-92ad-f1ae5fc78298 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:24.665652 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9886c-2d10-4d71-92ad-f1ae5fc78298 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:24.666476 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9886c-2d10-4d71-92ad-f1ae5fc78298 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:24.667850 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 91/182] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:24.668367 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b9886c-2d10-4d71-92ad-f1ae5fc78298 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:24.676661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-25285cfe-de6f-43fd-a179-0a8f957aaf2c None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:24.677841 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-25285cfe-de6f-43fd-a179-0a8f957aaf2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:24.745552 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 92/183] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:24.748998 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82c3f4-93fb-4350-823c-2d0388c03de9 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:24.749531 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82c3f4-93fb-4350-823c-2d0388c03de9 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:24.750267 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82c3f4-93fb-4350-823c-2d0388c03de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:24.775723 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-3c82c3f4-93fb-4350-823c-2d0388c03de9 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:24.777704 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-3c82c3f4-93fb-4350-823c-2d0388c03de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:24.788051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b9886c-2d10-4d71-92ad-f1ae5fc78298 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:24.840569 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dadb767-19c0-498b-8549-5ad014554399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:24.841241 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dadb767-19c0-498b-8549-5ad014554399 None admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:24.841519 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dadb767-19c0-498b-8549-5ad014554399 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:24.841796 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dadb767-19c0-498b-8549-5ad014554399 None admin] PATH_INFO: `/v3/projects/a7eea203d96a471cb590c418bb060d23/users/44e73f0cd35648198c204a3f657edccb/roles/651d60b8bf604682a0609ba8ad8c3f1b` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:24.844353 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dadb767-19c0-498b-8549-5ad014554399 None admin] RBAC: Authorizing `identity:create_grant(project_id=a7eea203d96a471cb590c418bb060d23, user_id=44e73f0cd35648198c204a3f657edccb, role_id=651d60b8bf604682a0609ba8ad8c3f1b)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:24.845226 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dadb767-19c0-498b-8549-5ad014554399 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:24.862189 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 92/184] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 14:12:24 2022] PUT /identity/v3/projects/a7eea203d96a471cb590c418bb060d23/users/44e73f0cd35648198c204a3f657edccb/roles/651d60b8bf604682a0609ba8ad8c3f1b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:12:24.867812 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dadb767-19c0-498b-8549-5ad014554399 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:24.905898 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 93/185] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:24.930511 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c82c3f4-93fb-4350-823c-2d0388c03de9 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:24.939479 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:24.939893 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:25.007326 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dadb767-19c0-498b-8549-5ad014554399 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:25.054053 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f6808-8682-402c-931a-bf57f87fecff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:25.054683 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f6808-8682-402c-931a-bf57f87fecff None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:25.054992 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f6808-8682-402c-931a-bf57f87fecff None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:25.055263 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f6808-8682-402c-931a-bf57f87fecff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:25.056142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57f6808-8682-402c-931a-bf57f87fecff None admin] RBAC: Adding query filter params (role.id=640af5182e6848498c16a19cddbfb738, scope.project.id=f7af21ee123e483ea1be1cb0e7f9f73b, user.id=75510b26c21742e091c6eca468df1c06) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:25.056525 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57f6808-8682-402c-931a-bf57f87fecff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:25.058019 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57f6808-8682-402c-931a-bf57f87fecff None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:25.064466 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 93/186] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:12:24 2022] GET /identity/v3/role_assignments?user.id=75510b26c21742e091c6eca468df1c06&scope.project.id=f7af21ee123e483ea1be1cb0e7f9f73b&role.id=640af5182e6848498c16a19cddbfb738 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:25.071019 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f6808-8682-402c-931a-bf57f87fecff None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:25.113040 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c82c3f4-93fb-4350-823c-2d0388c03de9 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:25.176311 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acff5b3-6922-47c8-9e41-c12fa191877a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:25.176975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acff5b3-6922-47c8-9e41-c12fa191877a None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:25.177255 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acff5b3-6922-47c8-9e41-c12fa191877a None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:25.177529 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acff5b3-6922-47c8-9e41-c12fa191877a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:25.194488 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acff5b3-6922-47c8-9e41-c12fa191877a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:25.200391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-4acff5b3-6922-47c8-9e41-c12fa191877a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:12:25.200391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:25.200391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:25.200391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:25.200391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:25.200391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:25.200391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:25.200391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:25.200391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:25.200391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:25.200391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:25.200391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:25.200391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:25.200391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:25.200391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:25.200391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:25.201323 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:25.201323 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:25.201323 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:25.201323 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:25.201323 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:25.201323 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:25.201323 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:25.201323 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:25.201323 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:25.201323 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:25.201323 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:25.201323 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:25.201323 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:25.201323 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:25.201323 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:25.201323 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:25.202751 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:25.202751 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:25.202751 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:25.202751 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:25.202751 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:25.202751 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:25.202751 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:25.202751 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:25.202751 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:25.202751 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:25.202751 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:12:25.202751 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:12:25.202751 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 94/187] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 14:12:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:12:25.207535 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acff5b3-6922-47c8-9e41-c12fa191877a None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:25.234523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f6808-8682-402c-931a-bf57f87fecff None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:25.246729 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:25.282508 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6350df5-d0cc-44b7-91f2-a0710fc1be44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:25.283499 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6350df5-d0cc-44b7-91f2-a0710fc1be44 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:25.284050 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6350df5-d0cc-44b7-91f2-a0710fc1be44 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:25.284598 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6350df5-d0cc-44b7-91f2-a0710fc1be44 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:25.287341 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6350df5-d0cc-44b7-91f2-a0710fc1be44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:25.288069 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6350df5-d0cc-44b7-91f2-a0710fc1be44 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:25.290325 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 94/188] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:12:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:25.296534 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6350df5-d0cc-44b7-91f2-a0710fc1be44 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:25.312068 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:25.360013 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acff5b3-6922-47c8-9e41-c12fa191877a None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:25.408044 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f2d8a5-f695-4394-b6b4-2f9915764e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:25.408660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2d8a5-f695-4394-b6b4-2f9915764e5a None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:25.408937 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2d8a5-f695-4394-b6b4-2f9915764e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:25.409208 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2d8a5-f695-4394-b6b4-2f9915764e5a None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:25.409885 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f2d8a5-f695-4394-b6b4-2f9915764e5a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:25.410185 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f2d8a5-f695-4394-b6b4-2f9915764e5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:25.413369 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f2d8a5-f695-4394-b6b4-2f9915764e5a None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:25.422747 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 95/189] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:12:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:25.423784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f2d8a5-f695-4394-b6b4-2f9915764e5a None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:25.424749 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6350df5-d0cc-44b7-91f2-a0710fc1be44 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:25.466514 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581d992-9735-428c-a25a-8bf248613b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:25.467161 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581d992-9735-428c-a25a-8bf248613b84 None admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:25.467441 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581d992-9735-428c-a25a-8bf248613b84 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:25.467713 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581d992-9735-428c-a25a-8bf248613b84 None admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:25.468366 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8581d992-9735-428c-a25a-8bf248613b84 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:25.469909 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8581d992-9735-428c-a25a-8bf248613b84 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:25.498852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 95/190] 10.222.0.52 () {60 vars in 1132 bytes} [Sat Sep 24 14:12:25 2022] POST /identity/v3/users => generated 297 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:12:25.498852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581d992-9735-428c-a25a-8bf248613b84 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:25.535427 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:25.565825 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f2d8a5-f695-4394-b6b4-2f9915764e5a None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:25.603960 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:25.614272 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3332b09c-7d9d-4ba4-a536-9f923934eb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:25.615026 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332b09c-7d9d-4ba4-a536-9f923934eb18 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:25.615307 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332b09c-7d9d-4ba4-a536-9f923934eb18 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:25.615580 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332b09c-7d9d-4ba4-a536-9f923934eb18 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:25.631873 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3332b09c-7d9d-4ba4-a536-9f923934eb18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:25.632329 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581d992-9735-428c-a25a-8bf248613b84 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:25.639361 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-3332b09c-7d9d-4ba4-a536-9f923934eb18 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:12:25.639361 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:25.639361 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:25.639361 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:25.639361 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:25.639361 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:25.639361 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:25.639361 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:25.639361 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:25.639361 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:25.639361 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:25.639361 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:25.639361 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:25.639361 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:25.639361 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:25.639361 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:25.640300 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:25.640300 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:25.640300 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:25.640300 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:25.640300 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:25.640300 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:25.640300 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:25.640300 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:25.640300 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:25.640300 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:25.640300 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:25.640300 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:25.640300 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:25.640300 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:25.640300 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:25.640300 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:25.641509 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:25.641509 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:25.641509 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:25.641509 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:25.641509 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:25.641509 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:25.641509 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:25.641509 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:25.641509 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:25.641509 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:25.641509 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:12:25.641509 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:12:25.641509 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 96/191] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 14:12:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:12:25.647095 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3332b09c-7d9d-4ba4-a536-9f923934eb18 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:25.684109 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a858f0-16dc-48cb-a9a5-dd4d1e9c79fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:25.684480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a858f0-16dc-48cb-a9a5-dd4d1e9c79fc None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:25.684751 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a858f0-16dc-48cb-a9a5-dd4d1e9c79fc None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:25.685022 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a858f0-16dc-48cb-a9a5-dd4d1e9c79fc None admin] PATH_INFO: `/v3/users/0fedb5ca122646508daedbbe1d46a9b0` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:25.702765 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a858f0-16dc-48cb-a9a5-dd4d1e9c79fc None admin] RBAC: Authorizing `identity:get_user(user_id=0fedb5ca122646508daedbbe1d46a9b0)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:25.704279 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a858f0-16dc-48cb-a9a5-dd4d1e9c79fc None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:25.705980 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 96/192] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:12:25 2022] GET /identity/v3/users/0fedb5ca122646508daedbbe1d46a9b0 => generated 283 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:25.713478 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a858f0-16dc-48cb-a9a5-dd4d1e9c79fc None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:25.780374 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3332b09c-7d9d-4ba4-a536-9f923934eb18 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:25.831244 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d6785-03be-47f2-b182-875f1684228b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:25.831898 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d6785-03be-47f2-b182-875f1684228b None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:25.832175 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d6785-03be-47f2-b182-875f1684228b None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:25.832444 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d6785-03be-47f2-b182-875f1684228b None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:25.833127 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65d6785-03be-47f2-b182-875f1684228b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:25.833460 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65d6785-03be-47f2-b182-875f1684228b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:25.835859 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65d6785-03be-47f2-b182-875f1684228b None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:25.845728 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:25.848815 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 97/193] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:12:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:25.854646 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d6785-03be-47f2-b182-875f1684228b None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:25.860631 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a858f0-16dc-48cb-a9a5-dd4d1e9c79fc None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:25.907655 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6651fd1d-0238-4ae3-96fa-c083393c7b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:25.908610 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651fd1d-0238-4ae3-96fa-c083393c7b37 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:25.909085 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651fd1d-0238-4ae3-96fa-c083393c7b37 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:25.909570 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651fd1d-0238-4ae3-96fa-c083393c7b37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:25.925364 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:25.978147 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d6785-03be-47f2-b182-875f1684228b None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:25.992055 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6651fd1d-0238-4ae3-96fa-c083393c7b37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:25.993773 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6651fd1d-0238-4ae3-96fa-c083393c7b37 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:26.031411 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b6b7d-a3a5-4b12-92e3-b6818ebb1052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:26.032550 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b6b7d-a3a5-4b12-92e3-b6818ebb1052 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:26.033348 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b6b7d-a3a5-4b12-92e3-b6818ebb1052 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:26.034041 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b6b7d-a3a5-4b12-92e3-b6818ebb1052 None admin] PATH_INFO: `/v3/users/0fedb5ca122646508daedbbe1d46a9b0` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:26.035388 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892b6b7d-a3a5-4b12-92e3-b6818ebb1052 None admin] RBAC: Authorizing `identity:get_user(user_id=0fedb5ca122646508daedbbe1d46a9b0)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:26.038188 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892b6b7d-a3a5-4b12-92e3-b6818ebb1052 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:26.039985 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 98/194] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:12:25 2022] GET /identity/v3/users/0fedb5ca122646508daedbbe1d46a9b0 => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:26.046098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b6b7d-a3a5-4b12-92e3-b6818ebb1052 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:26.104527 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:26.120675 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 97/195] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:12:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 24 14:12:26.128296 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6651fd1d-0238-4ae3-96fa-c083393c7b37 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:26.181580 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b6b7d-a3a5-4b12-92e3-b6818ebb1052 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:26.217032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:26.256036 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1ba32f-18a9-40c4-af14-f4a6d7f5e250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:26.257042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ba32f-18a9-40c4-af14-f4a6d7f5e250 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:26.257521 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ba32f-18a9-40c4-af14-f4a6d7f5e250 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:26.258028 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ba32f-18a9-40c4-af14-f4a6d7f5e250 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:26.288789 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6651fd1d-0238-4ae3-96fa-c083393c7b37 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:26.332489 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e499875-6649-467d-a125-42696717dfd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:26.335059 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e499875-6649-467d-a125-42696717dfd1 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:26.335345 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e499875-6649-467d-a125-42696717dfd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:26.335613 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e499875-6649-467d-a125-42696717dfd1 None admin] PATH_INFO: `/v3/projects/c73b2a612ca646969294ac368768fcb5` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:26.336819 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e499875-6649-467d-a125-42696717dfd1 None admin] RBAC: Authorizing `identity:get_project(project_id=c73b2a612ca646969294ac368768fcb5)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:26.338305 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e499875-6649-467d-a125-42696717dfd1 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:26.340261 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 98/196] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:12:26 2022] GET /identity/v3/projects/c73b2a612ca646969294ac368768fcb5 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:26.347333 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e499875-6649-467d-a125-42696717dfd1 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:26.395309 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1ba32f-18a9-40c4-af14-f4a6d7f5e250 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:26.396819 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1ba32f-18a9-40c4-af14-f4a6d7f5e250 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:26.450726 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:26.507144 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e499875-6649-467d-a125-42696717dfd1 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:26.545239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e87c4f-ae16-4230-b03d-bc2c1717d020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:26.545849 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e87c4f-ae16-4230-b03d-bc2c1717d020 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:26.546126 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e87c4f-ae16-4230-b03d-bc2c1717d020 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:26.546393 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e87c4f-ae16-4230-b03d-bc2c1717d020 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:26.547203 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e87c4f-ae16-4230-b03d-bc2c1717d020 None admin] RBAC: Adding query filter params (role.id=d00b78c32dbf4cc7925a55bb621d8fcf, scope.project.id=c73b2a612ca646969294ac368768fcb5, user.id=0fedb5ca122646508daedbbe1d46a9b0) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:26.547515 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e87c4f-ae16-4230-b03d-bc2c1717d020 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:26.549123 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e87c4f-ae16-4230-b03d-bc2c1717d020 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:26.555182 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 99/197] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:12:26 2022] GET /identity/v3/role_assignments?user.id=0fedb5ca122646508daedbbe1d46a9b0&scope.project.id=c73b2a612ca646969294ac368768fcb5&role.id=d00b78c32dbf4cc7925a55bb621d8fcf => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:26.590585 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 99/198] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:12:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:12:26.596720 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e87c4f-ae16-4230-b03d-bc2c1717d020 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:26.708184 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e87c4f-ae16-4230-b03d-bc2c1717d020 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:26.733936 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:26.746585 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff25bd-f010-415a-8dc1-eac37b95b366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:26.747268 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff25bd-f010-415a-8dc1-eac37b95b366 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:26.747548 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff25bd-f010-415a-8dc1-eac37b95b366 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:26.747819 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff25bd-f010-415a-8dc1-eac37b95b366 None admin] PATH_INFO: `/v3/projects/c73b2a612ca646969294ac368768fcb5` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:26.749036 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff25bd-f010-415a-8dc1-eac37b95b366 None admin] RBAC: Authorizing `identity:get_project(project_id=c73b2a612ca646969294ac368768fcb5)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:26.750505 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff25bd-f010-415a-8dc1-eac37b95b366 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:26.752551 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 100/199] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:12:26 2022] GET /identity/v3/projects/c73b2a612ca646969294ac368768fcb5 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:26.761480 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1ba32f-18a9-40c4-af14-f4a6d7f5e250 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:26.799299 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:26.942164 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1ba32f-18a9-40c4-af14-f4a6d7f5e250 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:26.988830 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567616e-4672-4cf2-a98b-a43d6e799bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:26.989811 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567616e-4672-4cf2-a98b-a43d6e799bcf None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:26.990291 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567616e-4672-4cf2-a98b-a43d6e799bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:26.990774 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567616e-4672-4cf2-a98b-a43d6e799bcf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:26.991740 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9567616e-4672-4cf2-a98b-a43d6e799bcf None admin] RBAC: Adding query filter params (role.id=25257d00ad2640c2a57d257b0279155f, scope.project.id=c73b2a612ca646969294ac368768fcb5, user.id=0fedb5ca122646508daedbbe1d46a9b0) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:26.992250 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9567616e-4672-4cf2-a98b-a43d6e799bcf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:26.994441 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9567616e-4672-4cf2-a98b-a43d6e799bcf None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:27.000566 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 100/200] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:12:26 2022] GET /identity/v3/role_assignments?user.id=0fedb5ca122646508daedbbe1d46a9b0&scope.project.id=c73b2a612ca646969294ac368768fcb5&role.id=25257d00ad2640c2a57d257b0279155f => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:27.376580 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d85a3f-291a-432f-b1cd-98bf8b581ef0 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:27.377057 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d85a3f-291a-432f-b1cd-98bf8b581ef0 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:27.377057 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d85a3f-291a-432f-b1cd-98bf8b581ef0 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:27.378624 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 101/201] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:27.383528 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132fb30d-7622-4659-ad20-d6a67a9db17f None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:27.383814 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132fb30d-7622-4659-ad20-d6a67a9db17f None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:27.384086 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132fb30d-7622-4659-ad20-d6a67a9db17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:27.405093 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-132fb30d-7622-4659-ad20-d6a67a9db17f None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:27.407323 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-132fb30d-7622-4659-ad20-d6a67a9db17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:27.490744 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 101/202] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:27.847920 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d85a3f-291a-432f-b1cd-98bf8b581ef0 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:27.970559 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d85a3f-291a-432f-b1cd-98bf8b581ef0 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:28.015650 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1aa53-1856-422b-adf9-f81ac6686daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:28.016303 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1aa53-1856-422b-adf9-f81ac6686daa None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:28.016584 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1aa53-1856-422b-adf9-f81ac6686daa None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:28.016856 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1aa53-1856-422b-adf9-f81ac6686daa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:28.031558 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d1aa53-1856-422b-adf9-f81ac6686daa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:28.036938 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-41d1aa53-1856-422b-adf9-f81ac6686daa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 14:12:28.036938 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:28.036938 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:28.036938 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:28.036938 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:28.036938 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:28.036938 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:28.036938 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:28.036938 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:28.036938 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:28.036938 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:28.036938 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:28.036938 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:28.036938 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:28.036938 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:28.036938 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:28.037866 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:28.037866 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:28.037866 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:28.037866 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:28.037866 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:28.037866 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:28.037866 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:28.037866 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:28.037866 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:28.037866 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:28.037866 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:28.037866 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:28.037866 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:28.037866 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:28.037866 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:28.037866 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:28.038627 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:28.038627 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:28.038627 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:28.038627 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:28.038627 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:28.038627 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:28.038627 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:28.038627 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:28.038627 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:28.038627 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:28.038627 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 14:12:28.038627 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:12:28.038627 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 102/203] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:12:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:12:28.047926 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132fb30d-7622-4659-ad20-d6a67a9db17f None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:28.069338 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:28.190201 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132fb30d-7622-4659-ad20-d6a67a9db17f None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:28.238273 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f2750-db73-475f-97a3-8ff53b7971a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:28.238923 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f2750-db73-475f-97a3-8ff53b7971a2 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:28.239204 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f2750-db73-475f-97a3-8ff53b7971a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:28.239474 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f2750-db73-475f-97a3-8ff53b7971a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:28.240149 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622f2750-db73-475f-97a3-8ff53b7971a2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:28.240451 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622f2750-db73-475f-97a3-8ff53b7971a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:28.241850 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622f2750-db73-475f-97a3-8ff53b7971a2 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:28.255083 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 102/204] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 14:12:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:28.260533 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1aa53-1856-422b-adf9-f81ac6686daa None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:28.322533 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:28.391428 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1aa53-1856-422b-adf9-f81ac6686daa None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:28.447657 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a2467-6d5d-4801-9cd7-429b4d839e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:28.449360 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd7e91-0e5f-4db3-84a0-fd0cc0a9d467 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:28.449801 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd7e91-0e5f-4db3-84a0-fd0cc0a9d467 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:28.449956 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd7e91-0e5f-4db3-84a0-fd0cc0a9d467 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:28.451394 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 103/205] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:28.451839 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a2467-6d5d-4801-9cd7-429b4d839e4f None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:28.452123 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a2467-6d5d-4801-9cd7-429b4d839e4f None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:28.452396 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a2467-6d5d-4801-9cd7-429b4d839e4f None admin] PATH_INFO: `/v3/users/44e73f0cd35648198c204a3f657edccb` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:28.453601 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328a2467-6d5d-4801-9cd7-429b4d839e4f None admin] RBAC: Authorizing `identity:get_user(user_id=44e73f0cd35648198c204a3f657edccb)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:28.455087 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328a2467-6d5d-4801-9cd7-429b4d839e4f None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:28.457574 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ec751-8c73-45a7-a929-897c4bc6f351 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:28.457686 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 103/206] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:12:28 2022] GET /identity/v3/users/44e73f0cd35648198c204a3f657edccb => generated 289 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:28.457759 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ec751-8c73-45a7-a929-897c4bc6f351 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:28.457759 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ec751-8c73-45a7-a929-897c4bc6f351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:28.461928 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a2467-6d5d-4801-9cd7-429b4d839e4f None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:28.492073 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-439ec751-8c73-45a7-a929-897c4bc6f351 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:28.493286 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-439ec751-8c73-45a7-a929-897c4bc6f351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:28.592092 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a2467-6d5d-4801-9cd7-429b4d839e4f None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:28.595699 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 104/207] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:28.597099 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda7801-c6e8-4940-a344-f5eb0a9e93ad None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:28.597386 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda7801-c6e8-4940-a344-f5eb0a9e93ad None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:28.597661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda7801-c6e8-4940-a344-f5eb0a9e93ad None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:28.599071 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 105/208] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:28.604264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228929d2-2918-46b1-8431-968e64c5028d None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:28.604549 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228929d2-2918-46b1-8431-968e64c5028d None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:28.604824 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228929d2-2918-46b1-8431-968e64c5028d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:28.617524 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:28.621362 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:28.641878 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-228929d2-2918-46b1-8431-968e64c5028d None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:28.643143 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-228929d2-2918-46b1-8431-968e64c5028d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:28.652362 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eed0d1-cc24-4ea4-9eac-dc97c44c30a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:28.652999 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eed0d1-cc24-4ea4-9eac-dc97c44c30a3 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:28.653722 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eed0d1-cc24-4ea4-9eac-dc97c44c30a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:28.654101 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eed0d1-cc24-4ea4-9eac-dc97c44c30a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:28.747418 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eed0d1-cc24-4ea4-9eac-dc97c44c30a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:28.749014 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eed0d1-cc24-4ea4-9eac-dc97c44c30a3 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:28.770369 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 106/209] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:12:28.871729 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 104/210] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:12:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 24 14:12:28.877084 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228929d2-2918-46b1-8431-968e64c5028d None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:28.992794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228929d2-2918-46b1-8431-968e64c5028d None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:29.008531 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eed0d1-cc24-4ea4-9eac-dc97c44c30a3 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:29.036107 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6efcc-bb57-4956-b6f4-6e2ed5a7b28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:29.036753 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6efcc-bb57-4956-b6f4-6e2ed5a7b28c None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:29.037032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6efcc-bb57-4956-b6f4-6e2ed5a7b28c None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:29.037305 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6efcc-bb57-4956-b6f4-6e2ed5a7b28c None admin] PATH_INFO: `/v3/projects/a7eea203d96a471cb590c418bb060d23` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:29.039107 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:29.039242 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f6efcc-bb57-4956-b6f4-6e2ed5a7b28c None admin] RBAC: Authorizing `identity:get_project(project_id=a7eea203d96a471cb590c418bb060d23)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:29.040645 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f6efcc-bb57-4956-b6f4-6e2ed5a7b28c None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:29.042347 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 107/211] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:12:28 2022] GET /identity/v3/projects/a7eea203d96a471cb590c418bb060d23 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:29.052081 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6efcc-bb57-4956-b6f4-6e2ed5a7b28c None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:29.058040 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:29.169712 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eed0d1-cc24-4ea4-9eac-dc97c44c30a3 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:29.208805 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6efcc-bb57-4956-b6f4-6e2ed5a7b28c None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:29.214935 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce4d04-2315-4edc-9bcd-552d583830e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:29.215653 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce4d04-2315-4edc-9bcd-552d583830e0 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:29.216004 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce4d04-2315-4edc-9bcd-552d583830e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:29.216340 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce4d04-2315-4edc-9bcd-552d583830e0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:29.230384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ce4d04-2315-4edc-9bcd-552d583830e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:29.237237 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-02ce4d04-2315-4edc-9bcd-552d583830e0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:12:29.237237 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:29.237237 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:29.237237 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:29.237237 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:29.237237 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:29.237237 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:29.237237 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:29.237237 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:29.237237 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:29.237237 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:29.237237 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:29.237237 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:29.237237 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:29.237237 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:29.237237 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:29.238105 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:29.238105 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:29.238105 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:29.238105 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:29.238105 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:29.238105 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:29.238105 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:29.238105 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:29.238105 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:29.238105 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:29.238105 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:29.238105 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:29.238105 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:29.238105 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:29.238105 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:29.238105 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:29.238895 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:29.238895 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:29.238895 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:29.238895 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:29.238895 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:29.238895 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:29.238895 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:29.238895 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:29.238895 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:29.238895 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:29.238895 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:12:29.238895 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:12:29.238895 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 105/212] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:12:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:12:29.242284 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce4d04-2315-4edc-9bcd-552d583830e0 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:29.247467 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a22575-0f79-4ac8-9aa7-3644f5372f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:29.248072 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a22575-0f79-4ac8-9aa7-3644f5372f5f None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:29.248350 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a22575-0f79-4ac8-9aa7-3644f5372f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:29.248622 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a22575-0f79-4ac8-9aa7-3644f5372f5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:29.249358 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a22575-0f79-4ac8-9aa7-3644f5372f5f None admin] RBAC: Adding query filter params (role.id=651d60b8bf604682a0609ba8ad8c3f1b, scope.project.id=a7eea203d96a471cb590c418bb060d23, user.id=44e73f0cd35648198c204a3f657edccb) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:29.249665 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a22575-0f79-4ac8-9aa7-3644f5372f5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:29.251139 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a22575-0f79-4ac8-9aa7-3644f5372f5f None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:29.257506 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 108/213] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:12:29 2022] GET /identity/v3/role_assignments?user.id=44e73f0cd35648198c204a3f657edccb&scope.project.id=a7eea203d96a471cb590c418bb060d23&role.id=651d60b8bf604682a0609ba8ad8c3f1b => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:29.258300 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a22575-0f79-4ac8-9aa7-3644f5372f5f None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:29.387593 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:29.404904 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:29.437049 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce4d04-2315-4edc-9bcd-552d583830e0 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:29.467937 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a22575-0f79-4ac8-9aa7-3644f5372f5f None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:29.504834 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130cdbef-1394-4a3a-a5ed-4128212529da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:29.505501 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130cdbef-1394-4a3a-a5ed-4128212529da None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:29.505781 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130cdbef-1394-4a3a-a5ed-4128212529da None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:29.506056 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130cdbef-1394-4a3a-a5ed-4128212529da None admin] PATH_INFO: `/v3/domains/default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:29.511119 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130cdbef-1394-4a3a-a5ed-4128212529da None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:29.512994 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130cdbef-1394-4a3a-a5ed-4128212529da None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:29.516065 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 106/214] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:12:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:29.522409 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130cdbef-1394-4a3a-a5ed-4128212529da None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:29.533720 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cd1b0a-4c95-4d83-a13f-aaa28a46af93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:29.534369 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd1b0a-4c95-4d83-a13f-aaa28a46af93 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:29.534651 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd1b0a-4c95-4d83-a13f-aaa28a46af93 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:29.534959 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd1b0a-4c95-4d83-a13f-aaa28a46af93 None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:29.535673 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cd1b0a-4c95-4d83-a13f-aaa28a46af93 None admin] RBAC: Adding query filter params (name=member) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:29.536024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cd1b0a-4c95-4d83-a13f-aaa28a46af93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:29.537499 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cd1b0a-4c95-4d83-a13f-aaa28a46af93 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:29.547704 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 109/215] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 14:12:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:29.552773 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cd1b0a-4c95-4d83-a13f-aaa28a46af93 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:29.653434 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130cdbef-1394-4a3a-a5ed-4128212529da None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:29.669111 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cd1b0a-4c95-4d83-a13f-aaa28a46af93 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:29.687498 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c3b760-9f09-4e58-a595-3611f7e624de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:29.688104 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c3b760-9f09-4e58-a595-3611f7e624de None admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:29.688381 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c3b760-9f09-4e58-a595-3611f7e624de None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:29.688651 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c3b760-9f09-4e58-a595-3611f7e624de None admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:29.689303 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c3b760-9f09-4e58-a595-3611f7e624de None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:29.690779 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c3b760-9f09-4e58-a595-3611f7e624de None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:29.695132 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:29.703922 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb03799-2bfc-45e4-adf1-434444490c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:29.704672 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb03799-2bfc-45e4-adf1-434444490c2e None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:29.704982 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb03799-2bfc-45e4-adf1-434444490c2e None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:29.705269 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb03799-2bfc-45e4-adf1-434444490c2e None admin] PATH_INFO: `/v3/users/cb791a7f757b4646bf3647ea3069d7eb` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:29.711537 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 107/216] 10.222.0.52 () {60 vars in 1132 bytes} [Sat Sep 24 14:12:29 2022] POST /identity/v3/users => generated 297 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:12:29.712809 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc5d0d-3b03-4e41-a632-9f5194064ffa None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:29.713096 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc5d0d-3b03-4e41-a632-9f5194064ffa None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:29.713369 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc5d0d-3b03-4e41-a632-9f5194064ffa None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:29.714592 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 108/217] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:29.715678 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137bc53-c41d-4179-a486-89f7905e8308 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:29.715961 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137bc53-c41d-4179-a486-89f7905e8308 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:29.716235 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137bc53-c41d-4179-a486-89f7905e8308 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:29.717648 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 109/218] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:29.720094 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfa564-df90-4d0d-a117-bb40d615d3d3 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:29.720498 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfa564-df90-4d0d-a117-bb40d615d3d3 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:29.720874 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfa564-df90-4d0d-a117-bb40d615d3d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:29.727631 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:29.731844 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb03799-2bfc-45e4-adf1-434444490c2e None admin] RBAC: Authorizing `identity:get_user(user_id=cb791a7f757b4646bf3647ea3069d7eb)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:29.733677 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb03799-2bfc-45e4-adf1-434444490c2e None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:29.740763 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 110/219] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:12:29 2022] GET /identity/v3/users/cb791a7f757b4646bf3647ea3069d7eb => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:29.742515 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457f4a9-66c4-409b-84dd-1de0a56b5579 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:29.742991 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457f4a9-66c4-409b-84dd-1de0a56b5579 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:29.743177 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-58dfa564-df90-4d0d-a117-bb40d615d3d3 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:29.744501 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457f4a9-66c4-409b-84dd-1de0a56b5579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:29.746098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-58dfa564-df90-4d0d-a117-bb40d615d3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:29.763789 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-c457f4a9-66c4-409b-84dd-1de0a56b5579 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:29.765249 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-c457f4a9-66c4-409b-84dd-1de0a56b5579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:29.828435 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 110/220] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:12:29.829160 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dfa564-df90-4d0d-a117-bb40d615d3d3 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:29.854979 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 111/221] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:29.950890 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dfa564-df90-4d0d-a117-bb40d615d3d3 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:29.980095 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:29.985732 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad56bb-d6b0-444a-87c5-3249825902b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:29.986529 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad56bb-d6b0-444a-87c5-3249825902b8 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:29.986870 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad56bb-d6b0-444a-87c5-3249825902b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:29.987106 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad56bb-d6b0-444a-87c5-3249825902b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:30.064068 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ad56bb-d6b0-444a-87c5-3249825902b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:30.065867 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ad56bb-d6b0-444a-87c5-3249825902b8 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:30.109219 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c457f4a9-66c4-409b-84dd-1de0a56b5579 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:30.177384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 111/222] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:12:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:12:30.178234 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad56bb-d6b0-444a-87c5-3249825902b8 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:30.212710 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:30.250062 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c457f4a9-66c4-409b-84dd-1de0a56b5579 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:30.288231 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e05fc59-d3ee-4537-8757-46cf1876cb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:30.288941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e05fc59-d3ee-4537-8757-46cf1876cb18 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:30.289224 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e05fc59-d3ee-4537-8757-46cf1876cb18 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:30.289527 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e05fc59-d3ee-4537-8757-46cf1876cb18 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:30.292683 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad56bb-d6b0-444a-87c5-3249825902b8 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:30.295208 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:30.301892 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e05fc59-d3ee-4537-8757-46cf1876cb18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:30.307735 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-9e05fc59-d3ee-4537-8757-46cf1876cb18 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:12:30.307735 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:30.307735 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:30.307735 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:30.307735 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:30.307735 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:30.307735 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:30.307735 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:30.307735 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:30.307735 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:30.307735 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:30.307735 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:30.307735 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:30.307735 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:30.307735 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:30.307735 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:30.308599 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:30.308599 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:30.308599 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:30.308599 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:30.308599 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:30.308599 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:30.308599 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:30.308599 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:30.308599 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:30.308599 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:30.308599 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:30.308599 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:30.308599 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:30.308599 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:30.308599 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:30.308599 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:30.309353 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:30.309353 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:30.309353 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:30.309353 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:30.309353 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:30.309353 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:30.309353 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:30.309353 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:30.309353 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:30.309353 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:30.309353 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:12:30.309353 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:12:30.309353 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 112/223] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 14:12:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:12:30.314854 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e05fc59-d3ee-4537-8757-46cf1876cb18 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:30.344800 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e17b524-f57e-4421-98c7-5b21ab4520e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:30.345933 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17b524-f57e-4421-98c7-5b21ab4520e8 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:30.346308 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17b524-f57e-4421-98c7-5b21ab4520e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:30.346932 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17b524-f57e-4421-98c7-5b21ab4520e8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:30.360181 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e17b524-f57e-4421-98c7-5b21ab4520e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:30.366542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-7e17b524-f57e-4421-98c7-5b21ab4520e8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:12:30.366542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:30.366542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:30.366542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:30.366542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:30.366542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:30.366542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:30.366542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:30.366542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:30.366542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:30.366542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:30.366542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:30.366542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:30.366542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:30.366542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:30.366542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:30.367700 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:30.367700 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:30.367700 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:30.367700 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:30.367700 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:30.367700 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:30.367700 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:30.367700 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:30.367700 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:30.367700 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:30.367700 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:30.367700 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:30.367700 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:30.367700 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:30.367700 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:30.367700 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:30.368679 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:30.368679 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:30.368679 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:30.368679 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:30.368679 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:30.368679 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:30.368679 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:30.368679 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:30.368679 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:30.368679 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:30.368679 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:12:30.368679 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:12:30.374878 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 112/224] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 14:12:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:12:30.375879 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e17b524-f57e-4421-98c7-5b21ab4520e8 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:30.446315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e05fc59-d3ee-4537-8757-46cf1876cb18 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:30.493670 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:30.498676 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7be3b-5e48-43ad-b875-766593b902fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:30.500135 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7be3b-5e48-43ad-b875-766593b902fd None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:30.500783 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7be3b-5e48-43ad-b875-766593b902fd None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:30.501459 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7be3b-5e48-43ad-b875-766593b902fd None admin] PATH_INFO: `/v3/projects/c73b2a612ca646969294ac368768fcb5` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:30.503365 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc7be3b-5e48-43ad-b875-766593b902fd None admin] RBAC: Authorizing `identity:get_project(project_id=c73b2a612ca646969294ac368768fcb5)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:30.505066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc7be3b-5e48-43ad-b875-766593b902fd None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:30.505593 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e17b524-f57e-4421-98c7-5b21ab4520e8 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:30.507505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 113/225] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:12:30 2022] GET /identity/v3/projects/c73b2a612ca646969294ac368768fcb5 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:30.508592 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7be3b-5e48-43ad-b875-766593b902fd None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:30.543247 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e75614-cb62-43e2-a2b0-9ab9b09c8a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:30.543856 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e75614-cb62-43e2-a2b0-9ab9b09c8a15 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:30.544130 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e75614-cb62-43e2-a2b0-9ab9b09c8a15 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:30.544441 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e75614-cb62-43e2-a2b0-9ab9b09c8a15 None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:30.545107 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e75614-cb62-43e2-a2b0-9ab9b09c8a15 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:30.545405 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e75614-cb62-43e2-a2b0-9ab9b09c8a15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:30.546779 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e75614-cb62-43e2-a2b0-9ab9b09c8a15 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:30.555380 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 113/226] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:12:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:30.556282 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e75614-cb62-43e2-a2b0-9ab9b09c8a15 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:30.564789 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:30.640620 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7be3b-5e48-43ad-b875-766593b902fd None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:30.668983 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e75614-cb62-43e2-a2b0-9ab9b09c8a15 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:30.689021 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b0e11-109d-4434-aaec-c022135b865c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:30.691285 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b0e11-109d-4434-aaec-c022135b865c None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:30.691810 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b0e11-109d-4434-aaec-c022135b865c None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:30.692590 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b0e11-109d-4434-aaec-c022135b865c None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:30.694137 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b0e11-109d-4434-aaec-c022135b865c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:30.694656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b0e11-109d-4434-aaec-c022135b865c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:30.697410 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b0e11-109d-4434-aaec-c022135b865c None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:30.703711 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d04ea-b49c-4d1d-a149-b293b349382a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:30.704319 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d04ea-b49c-4d1d-a149-b293b349382a None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:30.704592 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d04ea-b49c-4d1d-a149-b293b349382a None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:30.704860 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d04ea-b49c-4d1d-a149-b293b349382a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:30.705587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d04ea-b49c-4d1d-a149-b293b349382a None admin] RBAC: Adding query filter params (role.id=640af5182e6848498c16a19cddbfb738, scope.project.id=c73b2a612ca646969294ac368768fcb5, user.id=cb791a7f757b4646bf3647ea3069d7eb) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:30.705890 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d04ea-b49c-4d1d-a149-b293b349382a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:30.707362 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d04ea-b49c-4d1d-a149-b293b349382a None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:30.709463 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 114/227] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:12:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:30.711190 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b0e11-109d-4434-aaec-c022135b865c None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:30.713897 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 114/228] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:12:30 2022] GET /identity/v3/role_assignments?user.id=cb791a7f757b4646bf3647ea3069d7eb&scope.project.id=c73b2a612ca646969294ac368768fcb5&role.id=640af5182e6848498c16a19cddbfb738 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:30.719940 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d04ea-b49c-4d1d-a149-b293b349382a None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:30.768513 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:30.874406 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b0e11-109d-4434-aaec-c022135b865c None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:30.891722 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:30.922641 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18580d39-c5ce-4e38-8e44-80610e0aa4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:30.924533 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18580d39-c5ce-4e38-8e44-80610e0aa4b9 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:30.924816 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18580d39-c5ce-4e38-8e44-80610e0aa4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:30.925092 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18580d39-c5ce-4e38-8e44-80610e0aa4b9 None admin] PATH_INFO: `/v3/users/0fedb5ca122646508daedbbe1d46a9b0` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:30.926264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18580d39-c5ce-4e38-8e44-80610e0aa4b9 None admin] RBAC: Authorizing `identity:get_user(user_id=0fedb5ca122646508daedbbe1d46a9b0)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:30.927820 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18580d39-c5ce-4e38-8e44-80610e0aa4b9 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:30.929702 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 115/229] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:12:30 2022] GET /identity/v3/users/0fedb5ca122646508daedbbe1d46a9b0 => generated 283 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:30.936488 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18580d39-c5ce-4e38-8e44-80610e0aa4b9 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:30.995873 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d04ea-b49c-4d1d-a149-b293b349382a None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:31.057826 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18580d39-c5ce-4e38-8e44-80610e0aa4b9 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:31.079226 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484aff5b-3bd1-4194-b7a5-61f2b9eab96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:31.079871 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484aff5b-3bd1-4194-b7a5-61f2b9eab96c None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:31.080150 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484aff5b-3bd1-4194-b7a5-61f2b9eab96c None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:31.080421 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484aff5b-3bd1-4194-b7a5-61f2b9eab96c None admin] PATH_INFO: `/v3/users/0fedb5ca122646508daedbbe1d46a9b0` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:31.087553 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484aff5b-3bd1-4194-b7a5-61f2b9eab96c None admin] RBAC: Authorizing `identity:get_user(user_id=0fedb5ca122646508daedbbe1d46a9b0)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:31.089038 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484aff5b-3bd1-4194-b7a5-61f2b9eab96c None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:31.090620 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 115/230] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:12:30 2022] GET /identity/v3/users/0fedb5ca122646508daedbbe1d46a9b0 => generated 283 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:31.096986 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484aff5b-3bd1-4194-b7a5-61f2b9eab96c None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:31.099408 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a274687-fa4f-480f-b4a2-f1281617434d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:31.100391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a274687-fa4f-480f-b4a2-f1281617434d None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:31.100878 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a274687-fa4f-480f-b4a2-f1281617434d None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:31.101561 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a274687-fa4f-480f-b4a2-f1281617434d None admin] PATH_INFO: `/v3/projects/c73b2a612ca646969294ac368768fcb5` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:31.102903 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a274687-fa4f-480f-b4a2-f1281617434d None admin] RBAC: Authorizing `identity:get_project(project_id=c73b2a612ca646969294ac368768fcb5)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:31.106054 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a274687-fa4f-480f-b4a2-f1281617434d None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:31.107912 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 116/231] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:12:30 2022] GET /identity/v3/projects/c73b2a612ca646969294ac368768fcb5 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:31.114416 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a274687-fa4f-480f-b4a2-f1281617434d None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:31.182129 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:31.221174 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:31.252244 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484aff5b-3bd1-4194-b7a5-61f2b9eab96c None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:31.283039 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a274687-fa4f-480f-b4a2-f1281617434d None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:31.300117 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d3f92-dc1f-43d7-8752-dd919fd33a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:31.300761 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d3f92-dc1f-43d7-8752-dd919fd33a21 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:31.301089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d3f92-dc1f-43d7-8752-dd919fd33a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:31.301369 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d3f92-dc1f-43d7-8752-dd919fd33a21 None admin] PATH_INFO: `/v3/projects/c73b2a612ca646969294ac368768fcb5` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:31.302507 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5d3f92-dc1f-43d7-8752-dd919fd33a21 None admin] RBAC: Authorizing `identity:get_project(project_id=c73b2a612ca646969294ac368768fcb5)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:31.304004 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5d3f92-dc1f-43d7-8752-dd919fd33a21 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:31.305722 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 116/232] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:12:31 2022] GET /identity/v3/projects/c73b2a612ca646969294ac368768fcb5 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:31.311951 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d3f92-dc1f-43d7-8752-dd919fd33a21 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:31.358514 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34acd90c-6f2f-406d-9945-8ac8b50537af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:31.360613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34acd90c-6f2f-406d-9945-8ac8b50537af None admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:31.362199 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34acd90c-6f2f-406d-9945-8ac8b50537af None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:31.362649 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34acd90c-6f2f-406d-9945-8ac8b50537af None admin] PATH_INFO: `/v3/projects/c73b2a612ca646969294ac368768fcb5/users/0fedb5ca122646508daedbbe1d46a9b0/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:31.366243 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34acd90c-6f2f-406d-9945-8ac8b50537af None admin] RBAC: Authorizing `identity:create_grant(project_id=c73b2a612ca646969294ac368768fcb5, user_id=0fedb5ca122646508daedbbe1d46a9b0, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:31.371583 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34acd90c-6f2f-406d-9945-8ac8b50537af None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:31.403748 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 117/233] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 14:12:31 2022] PUT /identity/v3/projects/c73b2a612ca646969294ac368768fcb5/users/0fedb5ca122646508daedbbe1d46a9b0/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:12:31.464900 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d3f92-dc1f-43d7-8752-dd919fd33a21 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:31.501448 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d38cc-fa70-43ee-b3c0-ee129e7e2400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:31.502208 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d38cc-fa70-43ee-b3c0-ee129e7e2400 None admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:31.502405 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d38cc-fa70-43ee-b3c0-ee129e7e2400 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:31.502742 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d38cc-fa70-43ee-b3c0-ee129e7e2400 None admin] PATH_INFO: `/v3/projects/c73b2a612ca646969294ac368768fcb5/users/0fedb5ca122646508daedbbe1d46a9b0/roles/25257d00ad2640c2a57d257b0279155f` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:31.505080 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5d38cc-fa70-43ee-b3c0-ee129e7e2400 None admin] RBAC: Authorizing `identity:create_grant(project_id=c73b2a612ca646969294ac368768fcb5, user_id=0fedb5ca122646508daedbbe1d46a9b0, role_id=25257d00ad2640c2a57d257b0279155f)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:31.506623 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5d38cc-fa70-43ee-b3c0-ee129e7e2400 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:31.526340 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 117/234] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 14:12:31 2022] PUT /identity/v3/projects/c73b2a612ca646969294ac368768fcb5/users/0fedb5ca122646508daedbbe1d46a9b0/roles/25257d00ad2640c2a57d257b0279155f => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:12:32.472297 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb30310-b460-4309-9e90-84292318bcec None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:32.473339 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb30310-b460-4309-9e90-84292318bcec None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:32.473826 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb30310-b460-4309-9e90-84292318bcec None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:32.475656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 118/235] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:32.482057 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8cb451-5769-4b9d-8851-d24138f05906 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:32.482360 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8cb451-5769-4b9d-8851-d24138f05906 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:32.482689 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8cb451-5769-4b9d-8851-d24138f05906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:32.507443 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-2e8cb451-5769-4b9d-8851-d24138f05906 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:32.508699 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-2e8cb451-5769-4b9d-8851-d24138f05906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:32.522695 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:32.594840 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74fe02-c66b-4dbe-acb5-e00b7171780b None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:32.595121 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74fe02-c66b-4dbe-acb5-e00b7171780b None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:32.595395 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74fe02-c66b-4dbe-acb5-e00b7171780b None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:32.596780 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 119/236] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:32.603316 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84738b94-b1ad-4550-8307-a33e8c16d030 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:32.603752 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84738b94-b1ad-4550-8307-a33e8c16d030 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:32.604121 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84738b94-b1ad-4550-8307-a33e8c16d030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:32.619874 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 118/237] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:32.624492 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-84738b94-b1ad-4550-8307-a33e8c16d030 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:32.625774 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-84738b94-b1ad-4550-8307-a33e8c16d030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:32.697040 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 120/238] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:12:33.075022 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8cb451-5769-4b9d-8851-d24138f05906 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:33.086303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84738b94-b1ad-4550-8307-a33e8c16d030 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:33.143172 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:33.206474 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8cb451-5769-4b9d-8851-d24138f05906 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:33.227228 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84738b94-b1ad-4550-8307-a33e8c16d030 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:33.249859 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a0249-766a-4a02-818a-866deacdec7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:33.251174 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a0249-766a-4a02-818a-866deacdec7b None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:33.251461 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a0249-766a-4a02-818a-866deacdec7b None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:33.251730 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a0249-766a-4a02-818a-866deacdec7b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:33.263194 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a0249-766a-4a02-818a-866deacdec7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:33.266228 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0452d-3923-4afc-84f6-b5b460005a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:33.267156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0452d-3923-4afc-84f6-b5b460005a29 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:33.267633 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0452d-3923-4afc-84f6-b5b460005a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:33.268051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-6a5a0249-766a-4a02-818a-866deacdec7b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 14:12:33.268051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:33.268051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:33.268051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:33.268051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:33.268051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:33.268051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:33.268051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:33.268051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:33.268051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:33.268051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:33.268051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:33.268051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:33.268051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:33.268051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:33.268051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:33.268895 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:33.268895 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:33.268895 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:33.268895 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:33.268895 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:33.268895 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:33.268895 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:33.268895 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:33.268895 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:33.268895 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:33.268895 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:33.268895 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:33.268895 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:33.268895 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:33.268895 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:33.268895 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:33.269667 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:33.269667 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:33.269667 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:33.269667 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:33.269667 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:33.269667 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:33.269667 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:33.269667 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:33.269667 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:33.269667 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:33.269667 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 14:12:33.269667 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:12:33.270836 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0452d-3923-4afc-84f6-b5b460005a29 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:33.272217 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca0452d-3923-4afc-84f6-b5b460005a29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:33.273896 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 119/239] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:12:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:12:33.274005 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca0452d-3923-4afc-84f6-b5b460005a29 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:33.275198 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aeb86a-03ff-43f1-819f-075a09042df1 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:33.275486 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aeb86a-03ff-43f1-819f-075a09042df1 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:33.275761 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aeb86a-03ff-43f1-819f-075a09042df1 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:33.276478 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 121/240] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:12:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:33.284821 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 120/241] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:33.284821 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aeb86a-03ff-43f1-819f-075a09042df1 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:33.285021 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0452d-3923-4afc-84f6-b5b460005a29 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:33.289176 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:33.395745 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:33.398504 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0452d-3923-4afc-84f6-b5b460005a29 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:33.416774 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aeb86a-03ff-43f1-819f-075a09042df1 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:33.436170 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f01f1e3-88f5-493a-9e60-1847bc32dd72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:33.436910 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01f1e3-88f5-493a-9e60-1847bc32dd72 None admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:33.437287 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01f1e3-88f5-493a-9e60-1847bc32dd72 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:33.437684 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01f1e3-88f5-493a-9e60-1847bc32dd72 None admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:33.438424 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f01f1e3-88f5-493a-9e60-1847bc32dd72 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:33.439998 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f01f1e3-88f5-493a-9e60-1847bc32dd72 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:33.460330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b839878a-120f-4dec-b89d-9453b980829c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:33.461008 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839878a-120f-4dec-b89d-9453b980829c None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:33.461290 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839878a-120f-4dec-b89d-9453b980829c None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:33.461632 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839878a-120f-4dec-b89d-9453b980829c None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:33.462316 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b839878a-120f-4dec-b89d-9453b980829c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:33.462671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b839878a-120f-4dec-b89d-9453b980829c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:33.464557 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b839878a-120f-4dec-b89d-9453b980829c None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:33.477860 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 122/242] 10.222.0.52 () {60 vars in 1132 bytes} [Sat Sep 24 14:12:33 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:12:33.477860 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b71f26b-eaf8-45e8-9bef-0757a0d58b5a None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:33.477860 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b71f26b-eaf8-45e8-9bef-0757a0d58b5a None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:33.477860 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b71f26b-eaf8-45e8-9bef-0757a0d58b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:33.485065 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 121/243] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 14:12:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:33.497861 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b839878a-120f-4dec-b89d-9453b980829c None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:33.527982 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-8b71f26b-eaf8-45e8-9bef-0757a0d58b5a None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:33.529202 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-8b71f26b-eaf8-45e8-9bef-0757a0d58b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:33.614789 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:33.658744 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 123/244] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:33.670190 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b839878a-120f-4dec-b89d-9453b980829c None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:33.705537 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d22d7-e006-48c4-99ca-b98bdc0642da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:33.706172 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d22d7-e006-48c4-99ca-b98bdc0642da None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:33.706447 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d22d7-e006-48c4-99ca-b98bdc0642da None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:33.706720 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d22d7-e006-48c4-99ca-b98bdc0642da None admin] PATH_INFO: `/v3/users/fed403235e194e86adc513853e82c550` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:33.722095 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6d22d7-e006-48c4-99ca-b98bdc0642da None admin] RBAC: Authorizing `identity:get_user(user_id=fed403235e194e86adc513853e82c550)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:33.723593 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6d22d7-e006-48c4-99ca-b98bdc0642da None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:33.725627 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 122/245] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:12:33 2022] GET /identity/v3/users/fed403235e194e86adc513853e82c550 => generated 297 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:33.731918 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b71f26b-eaf8-45e8-9bef-0757a0d58b5a None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:33.843290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:33.855162 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b71f26b-eaf8-45e8-9bef-0757a0d58b5a None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:33.894420 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e93111e-b57a-4dbd-ac52-83091727b416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:33.895066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93111e-b57a-4dbd-ac52-83091727b416 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:33.895342 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93111e-b57a-4dbd-ac52-83091727b416 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:33.895608 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93111e-b57a-4dbd-ac52-83091727b416 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:33.914591 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d515931-cb4c-4f32-9bc4-04c55d450344 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:33.914878 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d515931-cb4c-4f32-9bc4-04c55d450344 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:33.915097 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d515931-cb4c-4f32-9bc4-04c55d450344 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:33.916974 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 123/246] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:12:33.926561 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3f2f2-f06a-4572-a826-b81e1f5ae2bb None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:33.927020 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3f2f2-f06a-4572-a826-b81e1f5ae2bb None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:33.927388 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3f2f2-f06a-4572-a826-b81e1f5ae2bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:33.951262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-84a3f2f2-f06a-4572-a826-b81e1f5ae2bb None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:33.952602 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-84a3f2f2-f06a-4572-a826-b81e1f5ae2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:33.981997 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e93111e-b57a-4dbd-ac52-83091727b416 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:33.983531 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e93111e-b57a-4dbd-ac52-83091727b416 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:34.046177 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 124/247] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:12:34.047790 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a3f2f2-f06a-4572-a826-b81e1f5ae2bb None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:34.093092 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:34.113726 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 124/248] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:12:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:12:34.119690 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e93111e-b57a-4dbd-ac52-83091727b416 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:34.121448 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:34.177979 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a3f2f2-f06a-4572-a826-b81e1f5ae2bb None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:34.229110 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90493909-3fee-42c7-821f-cef357c6252e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:34.229990 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90493909-3fee-42c7-821f-cef357c6252e None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:34.230434 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90493909-3fee-42c7-821f-cef357c6252e None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:34.231473 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90493909-3fee-42c7-821f-cef357c6252e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:34.247383 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90493909-3fee-42c7-821f-cef357c6252e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:34.249050 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e93111e-b57a-4dbd-ac52-83091727b416 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:34.253865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-90493909-3fee-42c7-821f-cef357c6252e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:12:34.253865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:34.253865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:34.253865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:34.253865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:34.253865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:34.253865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:34.253865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:34.253865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:34.253865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:34.253865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:34.253865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:34.253865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:34.253865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:34.253865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:34.253865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:34.255019 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:34.255019 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:34.255019 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:34.255019 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:34.255019 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:34.255019 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:34.255019 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:34.255019 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:34.255019 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:34.255019 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:34.255019 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:34.255019 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:34.255019 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:34.255019 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:34.255019 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:34.255019 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:34.256016 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:34.256016 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:34.256016 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:34.256016 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:34.256016 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:34.256016 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:34.256016 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:34.256016 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:34.256016 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:34.256016 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:34.256016 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:12:34.256016 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:12:34.257930 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 125/249] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:12:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:12:34.267228 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90493909-3fee-42c7-821f-cef357c6252e None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:34.305404 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c756b4e-1b07-4b58-8f60-7dca8d744637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:34.307333 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c756b4e-1b07-4b58-8f60-7dca8d744637 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:34.308376 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c756b4e-1b07-4b58-8f60-7dca8d744637 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:34.309137 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c756b4e-1b07-4b58-8f60-7dca8d744637 None admin] PATH_INFO: `/v3/projects/c73b2a612ca646969294ac368768fcb5` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:34.310979 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c756b4e-1b07-4b58-8f60-7dca8d744637 None admin] RBAC: Authorizing `identity:get_project(project_id=c73b2a612ca646969294ac368768fcb5)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:34.313171 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c756b4e-1b07-4b58-8f60-7dca8d744637 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:34.315554 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 125/250] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:12:34 2022] GET /identity/v3/projects/c73b2a612ca646969294ac368768fcb5 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:34.322415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c756b4e-1b07-4b58-8f60-7dca8d744637 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:34.398631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:34.412727 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90493909-3fee-42c7-821f-cef357c6252e None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:34.432343 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:34.455006 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c756b4e-1b07-4b58-8f60-7dca8d744637 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:34.459474 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a8b044-ff65-4000-b232-448937f06b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:34.460192 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a8b044-ff65-4000-b232-448937f06b1f None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:34.460560 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a8b044-ff65-4000-b232-448937f06b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:34.460923 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a8b044-ff65-4000-b232-448937f06b1f None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:34.461708 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a8b044-ff65-4000-b232-448937f06b1f None admin] RBAC: Adding query filter params (name=member) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:34.462104 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a8b044-ff65-4000-b232-448937f06b1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:34.463722 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a8b044-ff65-4000-b232-448937f06b1f None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:34.477937 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 126/251] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 14:12:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:34.479119 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a8b044-ff65-4000-b232-448937f06b1f None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:34.501509 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12772514-011c-4095-8257-8c917b77010b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:34.502689 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12772514-011c-4095-8257-8c917b77010b None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:34.503387 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12772514-011c-4095-8257-8c917b77010b None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:34.503805 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12772514-011c-4095-8257-8c917b77010b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:34.504888 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12772514-011c-4095-8257-8c917b77010b None admin] RBAC: Adding query filter params (role.id=651d60b8bf604682a0609ba8ad8c3f1b, scope.project.id=c73b2a612ca646969294ac368768fcb5, user.id=fed403235e194e86adc513853e82c550) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:34.505406 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12772514-011c-4095-8257-8c917b77010b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:34.507093 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12772514-011c-4095-8257-8c917b77010b None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:34.520542 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 126/252] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:12:34 2022] GET /identity/v3/role_assignments?user.id=fed403235e194e86adc513853e82c550&scope.project.id=c73b2a612ca646969294ac368768fcb5&role.id=651d60b8bf604682a0609ba8ad8c3f1b => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:34.528361 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0356b90b-8c8b-4539-9066-26a4b03de22e None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:34.528657 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0356b90b-8c8b-4539-9066-26a4b03de22e None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:34.528907 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0356b90b-8c8b-4539-9066-26a4b03de22e None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:34.530414 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 127/253] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:34 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:34.534868 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0356b90b-8c8b-4539-9066-26a4b03de22e None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:34.661671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a8b044-ff65-4000-b232-448937f06b1f None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:34.703323 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466b22e-ee68-443b-9f64-e66d86493211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:34.703886 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466b22e-ee68-443b-9f64-e66d86493211 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:34.704164 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466b22e-ee68-443b-9f64-e66d86493211 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:34.704431 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466b22e-ee68-443b-9f64-e66d86493211 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:34.716374 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8466b22e-ee68-443b-9f64-e66d86493211 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:34.722024 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-8466b22e-ee68-443b-9f64-e66d86493211 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:12:34.722024 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:34.722024 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:34.722024 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:34.722024 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:34.722024 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:34.722024 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:34.722024 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:34.722024 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:34.722024 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:34.722024 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:34.722024 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:34.722024 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:34.722024 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:34.722024 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:34.722024 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:34.723006 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:34.723006 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:34.723006 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:34.723006 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:34.723006 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:34.723006 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:34.723006 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:34.723006 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:34.723006 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:34.723006 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:34.723006 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:34.723006 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:34.723006 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:34.723006 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:34.723006 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:34.723006 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:34.723776 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:34.723776 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:34.723776 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:34.723776 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:34.723776 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:34.723776 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:34.723776 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:34.723776 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:34.723776 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:34.723776 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:34.723776 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:12:34.723776 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:12:34.723776 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 127/254] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 14:12:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:12:34.725134 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6922c11-5d0a-4673-88dc-ab2522b7ec71 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:34.725134 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6922c11-5d0a-4673-88dc-ab2522b7ec71 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:34.725419 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6922c11-5d0a-4673-88dc-ab2522b7ec71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:34.747335 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-b6922c11-5d0a-4673-88dc-ab2522b7ec71 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:34.748904 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-b6922c11-5d0a-4673-88dc-ab2522b7ec71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:34.756643 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0356b90b-8c8b-4539-9066-26a4b03de22e None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:34.777635 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:34.783475 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:34.799780 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e75ae3a-d016-48aa-8241-564a9dcb638e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:34.800567 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75ae3a-d016-48aa-8241-564a9dcb638e None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:34.800956 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75ae3a-d016-48aa-8241-564a9dcb638e None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:34.801299 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75ae3a-d016-48aa-8241-564a9dcb638e None admin] PATH_INFO: `/v3/users/cb791a7f757b4646bf3647ea3069d7eb` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:34.803524 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e75ae3a-d016-48aa-8241-564a9dcb638e None admin] RBAC: Authorizing `identity:get_user(user_id=cb791a7f757b4646bf3647ea3069d7eb)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:34.805265 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e75ae3a-d016-48aa-8241-564a9dcb638e None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:34.809039 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 128/255] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:12:34 2022] GET /identity/v3/users/cb791a7f757b4646bf3647ea3069d7eb => generated 297 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:34.809859 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e75ae3a-d016-48aa-8241-564a9dcb638e None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:34.828754 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 128/256] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:34.830033 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6922c11-5d0a-4673-88dc-ab2522b7ec71 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:35.005564 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e75ae3a-d016-48aa-8241-564a9dcb638e None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:35.011699 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6922c11-5d0a-4673-88dc-ab2522b7ec71 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:35.070115 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca2e5c3-264d-439a-ae80-77b2c9d8fdd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:35.071258 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2e5c3-264d-439a-ae80-77b2c9d8fdd4 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:35.071711 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2e5c3-264d-439a-ae80-77b2c9d8fdd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:35.072756 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2e5c3-264d-439a-ae80-77b2c9d8fdd4 None admin] PATH_INFO: `/v3/projects/c73b2a612ca646969294ac368768fcb5` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:35.074098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca2e5c3-264d-439a-ae80-77b2c9d8fdd4 None admin] RBAC: Authorizing `identity:get_project(project_id=c73b2a612ca646969294ac368768fcb5)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:35.075947 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca2e5c3-264d-439a-ae80-77b2c9d8fdd4 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:35.077764 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 129/257] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:12:34 2022] GET /identity/v3/projects/c73b2a612ca646969294ac368768fcb5 => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:35.083509 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca2e5c3-264d-439a-ae80-77b2c9d8fdd4 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:35.087290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59254aa8-8a92-48ac-b688-602400865961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:35.087960 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59254aa8-8a92-48ac-b688-602400865961 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:35.089019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59254aa8-8a92-48ac-b688-602400865961 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:35.089019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59254aa8-8a92-48ac-b688-602400865961 None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:35.089182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59254aa8-8a92-48ac-b688-602400865961 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:35.089462 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59254aa8-8a92-48ac-b688-602400865961 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:35.090891 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59254aa8-8a92-48ac-b688-602400865961 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:35.112999 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 129/258] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:12:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:35.118840 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59254aa8-8a92-48ac-b688-602400865961 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:35.125684 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:35.202013 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca2e5c3-264d-439a-ae80-77b2c9d8fdd4 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:35.238734 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13845231-02c6-4325-a0f6-e88ff2dca922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:35.239718 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13845231-02c6-4325-a0f6-e88ff2dca922 None admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:35.240165 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13845231-02c6-4325-a0f6-e88ff2dca922 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:35.240601 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13845231-02c6-4325-a0f6-e88ff2dca922 None admin] PATH_INFO: `/v3/projects/c73b2a612ca646969294ac368768fcb5/users/cb791a7f757b4646bf3647ea3069d7eb/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:35.242651 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13845231-02c6-4325-a0f6-e88ff2dca922 None admin] RBAC: Authorizing `identity:create_grant(project_id=c73b2a612ca646969294ac368768fcb5, user_id=cb791a7f757b4646bf3647ea3069d7eb, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:35.244792 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:35.245013 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13845231-02c6-4325-a0f6-e88ff2dca922 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:35.275670 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 130/259] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 14:12:35 2022] PUT /identity/v3/projects/c73b2a612ca646969294ac368768fcb5/users/cb791a7f757b4646bf3647ea3069d7eb/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:12:35.313407 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13845231-02c6-4325-a0f6-e88ff2dca922 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:35.338025 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59254aa8-8a92-48ac-b688-602400865961 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:35.379757 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35138a2-f83b-45d5-ba4c-79e599371735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:35.380400 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35138a2-f83b-45d5-ba4c-79e599371735 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:35.380683 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35138a2-f83b-45d5-ba4c-79e599371735 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:35.380961 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35138a2-f83b-45d5-ba4c-79e599371735 None admin] PATH_INFO: `/v3/users/0fedb5ca122646508daedbbe1d46a9b0` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:35.382108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35138a2-f83b-45d5-ba4c-79e599371735 None admin] RBAC: Authorizing `identity:get_user(user_id=0fedb5ca122646508daedbbe1d46a9b0)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:35.384193 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35138a2-f83b-45d5-ba4c-79e599371735 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:35.386680 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 130/260] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:12:35 2022] GET /identity/v3/users/0fedb5ca122646508daedbbe1d46a9b0 => generated 283 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:35.392453 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35138a2-f83b-45d5-ba4c-79e599371735 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:35.517974 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:35.537104 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13845231-02c6-4325-a0f6-e88ff2dca922 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:35.611551 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69f2b7-74fe-42cd-92be-d2e851e9b246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:35.612557 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69f2b7-74fe-42cd-92be-d2e851e9b246 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:35.613035 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69f2b7-74fe-42cd-92be-d2e851e9b246 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:35.613520 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69f2b7-74fe-42cd-92be-d2e851e9b246 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:35.624991 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc69f2b7-74fe-42cd-92be-d2e851e9b246 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:35.628719 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35138a2-f83b-45d5-ba4c-79e599371735 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:35.630799 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-dc69f2b7-74fe-42cd-92be-d2e851e9b246 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:12:35.630799 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:35.630799 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:35.630799 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:35.630799 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:35.630799 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:35.630799 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:35.630799 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:35.630799 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:35.630799 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:35.630799 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:35.630799 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:35.630799 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:35.630799 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:35.630799 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:35.630799 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:35.631662 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:35.631662 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:35.631662 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:35.631662 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:35.631662 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:35.631662 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:35.631662 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:35.631662 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:35.631662 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:35.631662 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:35.631662 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:35.631662 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:35.631662 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:35.631662 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:35.631662 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:35.631662 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:35.632502 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:35.632502 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:35.632502 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:35.632502 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:35.632502 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:35.632502 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:35.632502 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:35.632502 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:35.632502 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:35.632502 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:35.632502 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:12:35.632502 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:12:35.635598 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 131/261] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 14:12:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:12:35.641678 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69f2b7-74fe-42cd-92be-d2e851e9b246 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:35.649099 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:35.674572 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f8254e-b718-4df4-a120-87cbe5dbac14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:35.675216 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f8254e-b718-4df4-a120-87cbe5dbac14 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:35.675492 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f8254e-b718-4df4-a120-87cbe5dbac14 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:35.675758 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f8254e-b718-4df4-a120-87cbe5dbac14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:35.803716 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f8254e-b718-4df4-a120-87cbe5dbac14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:35.805110 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f8254e-b718-4df4-a120-87cbe5dbac14 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:35.841848 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69f2b7-74fe-42cd-92be-d2e851e9b246 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:35.892124 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4aa44-cae4-440f-9298-fcb890d70f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:35.892815 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4aa44-cae4-440f-9298-fcb890d70f67 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:35.893093 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4aa44-cae4-440f-9298-fcb890d70f67 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:35.893366 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4aa44-cae4-440f-9298-fcb890d70f67 None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:35.894035 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b4aa44-cae4-440f-9298-fcb890d70f67 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:35.894338 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b4aa44-cae4-440f-9298-fcb890d70f67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:35.895872 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b4aa44-cae4-440f-9298-fcb890d70f67 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:35.914297 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 132/262] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:12:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:35.921414 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4aa44-cae4-440f-9298-fcb890d70f67 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:35.955090 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:36.055418 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 131/263] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:12:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 663 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:12:36.061346 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f8254e-b718-4df4-a120-87cbe5dbac14 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:36.084632 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4aa44-cae4-440f-9298-fcb890d70f67 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:36.086443 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:36.166726 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f273c-9378-4f3a-a999-612315a83417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:36.167803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f273c-9378-4f3a-a999-612315a83417 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:36.168085 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f273c-9378-4f3a-a999-612315a83417 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:36.168353 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f273c-9378-4f3a-a999-612315a83417 None admin] PATH_INFO: `/v3/users/0fedb5ca122646508daedbbe1d46a9b0` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:36.179559 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114f273c-9378-4f3a-a999-612315a83417 None admin] RBAC: Authorizing `identity:get_user(user_id=0fedb5ca122646508daedbbe1d46a9b0)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:36.181018 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114f273c-9378-4f3a-a999-612315a83417 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:36.182890 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 133/264] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:12:35 2022] GET /identity/v3/users/0fedb5ca122646508daedbbe1d46a9b0 => generated 283 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:36.192663 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f273c-9378-4f3a-a999-612315a83417 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:36.234199 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f8254e-b718-4df4-a120-87cbe5dbac14 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:36.301110 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf1ff7-7024-4f86-94ba-65ca159325dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:36.301748 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf1ff7-7024-4f86-94ba-65ca159325dd None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:36.302023 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf1ff7-7024-4f86-94ba-65ca159325dd None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:36.302287 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf1ff7-7024-4f86-94ba-65ca159325dd None admin] PATH_INFO: `/v3/projects/c73b2a612ca646969294ac368768fcb5` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:36.303550 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bf1ff7-7024-4f86-94ba-65ca159325dd None admin] RBAC: Authorizing `identity:get_project(project_id=c73b2a612ca646969294ac368768fcb5)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:36.306299 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bf1ff7-7024-4f86-94ba-65ca159325dd None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:36.311782 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 132/265] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:12:36 2022] GET /identity/v3/projects/c73b2a612ca646969294ac368768fcb5 => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:36.318282 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf1ff7-7024-4f86-94ba-65ca159325dd None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:36.390991 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:36.405163 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f273c-9378-4f3a-a999-612315a83417 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:36.441938 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8445bcb-efe7-49b2-9a78-0ea49bc4d205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:36.442546 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8445bcb-efe7-49b2-9a78-0ea49bc4d205 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:36.442835 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8445bcb-efe7-49b2-9a78-0ea49bc4d205 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:36.443109 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8445bcb-efe7-49b2-9a78-0ea49bc4d205 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:36.486768 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:36.527003 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf1ff7-7024-4f86-94ba-65ca159325dd None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:36.532092 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8445bcb-efe7-49b2-9a78-0ea49bc4d205 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:36.533580 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8445bcb-efe7-49b2-9a78-0ea49bc4d205 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:36.608198 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bbe63b-915b-4a5d-a2fb-72939900b704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:36.608736 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbe63b-915b-4a5d-a2fb-72939900b704 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:36.609014 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbe63b-915b-4a5d-a2fb-72939900b704 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:36.609282 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbe63b-915b-4a5d-a2fb-72939900b704 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:36.610009 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bbe63b-915b-4a5d-a2fb-72939900b704 None admin] RBAC: Adding query filter params (role.id=d00b78c32dbf4cc7925a55bb621d8fcf, scope.project.id=c73b2a612ca646969294ac368768fcb5, user.id=0fedb5ca122646508daedbbe1d46a9b0) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:36.610312 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bbe63b-915b-4a5d-a2fb-72939900b704 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:36.611782 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bbe63b-915b-4a5d-a2fb-72939900b704 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:36.618317 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 133/266] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:12:36 2022] GET /identity/v3/role_assignments?user.id=0fedb5ca122646508daedbbe1d46a9b0&scope.project.id=c73b2a612ca646969294ac368768fcb5&role.id=d00b78c32dbf4cc7925a55bb621d8fcf => generated 608 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:36.651210 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 134/267] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:12:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:12:36.655355 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:36.657472 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bbe63b-915b-4a5d-a2fb-72939900b704 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:36.817035 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bbe63b-915b-4a5d-a2fb-72939900b704 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:36.840971 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9140f97-7d77-4239-be4a-17f4c88c3f9f None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:36.841244 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9140f97-7d77-4239-be4a-17f4c88c3f9f None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:36.841757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9140f97-7d77-4239-be4a-17f4c88c3f9f None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:36.844804 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 135/268] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:36.850352 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb99b8-f599-4492-8642-e53d09fbfe54 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:36.851141 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb99b8-f599-4492-8642-e53d09fbfe54 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:36.852328 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb99b8-f599-4492-8642-e53d09fbfe54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:36.869705 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5655b38-f6c2-4866-b398-27921e6bfd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:36.870279 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5655b38-f6c2-4866-b398-27921e6bfd5e None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:36.870561 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5655b38-f6c2-4866-b398-27921e6bfd5e None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:36.870870 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5655b38-f6c2-4866-b398-27921e6bfd5e None admin] PATH_INFO: `/v3/projects/c73b2a612ca646969294ac368768fcb5` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:36.874206 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5655b38-f6c2-4866-b398-27921e6bfd5e None admin] RBAC: Authorizing `identity:get_project(project_id=c73b2a612ca646969294ac368768fcb5)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:36.875749 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5655b38-f6c2-4866-b398-27921e6bfd5e None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:36.877496 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 134/269] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:12:36 2022] GET /identity/v3/projects/c73b2a612ca646969294ac368768fcb5 => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:36.886487 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-a6fb99b8-f599-4492-8642-e53d09fbfe54 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:36.886802 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5655b38-f6c2-4866-b398-27921e6bfd5e None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:36.888259 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-a6fb99b8-f599-4492-8642-e53d09fbfe54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:36.928632 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:36.978971 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 136/270] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:12:37.081559 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5655b38-f6c2-4866-b398-27921e6bfd5e None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:37.126423 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6327eef-f197-4c74-9aa0-a94c9fe56a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:37.126996 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6327eef-f197-4c74-9aa0-a94c9fe56a48 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:37.127280 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6327eef-f197-4c74-9aa0-a94c9fe56a48 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:37.127554 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6327eef-f197-4c74-9aa0-a94c9fe56a48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:37.128288 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6327eef-f197-4c74-9aa0-a94c9fe56a48 None admin] RBAC: Adding query filter params (role.id=25257d00ad2640c2a57d257b0279155f, scope.project.id=c73b2a612ca646969294ac368768fcb5, user.id=0fedb5ca122646508daedbbe1d46a9b0) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:37.128597 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6327eef-f197-4c74-9aa0-a94c9fe56a48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:37.130057 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6327eef-f197-4c74-9aa0-a94c9fe56a48 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:37.140065 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 135/271] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:12:36 2022] GET /identity/v3/role_assignments?user.id=0fedb5ca122646508daedbbe1d46a9b0&scope.project.id=c73b2a612ca646969294ac368768fcb5&role.id=25257d00ad2640c2a57d257b0279155f => generated 608 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:37.300422 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb99b8-f599-4492-8642-e53d09fbfe54 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:37.370882 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341f0af-f525-4ab1-b1ee-ca3a3c71a45c None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:37.371510 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341f0af-f525-4ab1-b1ee-ca3a3c71a45c None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:37.372013 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341f0af-f525-4ab1-b1ee-ca3a3c71a45c None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:37.373951 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 136/272] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:37.379615 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28975aa0-de56-4c74-b282-c67d10b29123 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:37.380234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28975aa0-de56-4c74-b282-c67d10b29123 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:37.380708 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28975aa0-de56-4c74-b282-c67d10b29123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:37.412600 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-28975aa0-de56-4c74-b282-c67d10b29123 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:37.414147 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-28975aa0-de56-4c74-b282-c67d10b29123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:37.434491 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb99b8-f599-4492-8642-e53d09fbfe54 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:37.459227 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:37.497170 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9a230-988d-48d1-87a2-b6fca2a9dde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:37.498242 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9a230-988d-48d1-87a2-b6fca2a9dde3 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:37.498717 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9a230-988d-48d1-87a2-b6fca2a9dde3 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:37.499209 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9a230-988d-48d1-87a2-b6fca2a9dde3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:37.512434 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9a230-988d-48d1-87a2-b6fca2a9dde3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:37.518671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-a3c9a230-988d-48d1-87a2-b6fca2a9dde3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:12:37.518671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:37.518671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:37.518671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:37.518671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:37.518671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:37.518671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:37.518671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:37.518671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:37.518671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:37.518671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:37.518671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:37.518671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:37.518671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:37.518671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:37.518671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:37.519555 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:37.519555 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:37.519555 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:37.519555 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:37.519555 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:37.519555 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:37.519555 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:37.519555 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:37.519555 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:37.519555 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:37.519555 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:37.519555 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:37.519555 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:37.519555 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:37.519555 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:37.519555 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:37.520316 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:37.520316 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:37.520316 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:37.520316 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:37.520316 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:37.520316 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:37.520316 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:37.520316 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:37.520316 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:37.520316 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:37.520316 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:12:37.520316 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:12:37.523223 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 137/273] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:12:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:12:37.529914 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9a230-988d-48d1-87a2-b6fca2a9dde3 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:37.533206 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 137/274] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:37.637829 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9a230-988d-48d1-87a2-b6fca2a9dde3 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:37.673027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c231edb-6dcf-4414-9419-a27242efbbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:37.673647 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c231edb-6dcf-4414-9419-a27242efbbb8 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:37.673928 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c231edb-6dcf-4414-9419-a27242efbbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:37.674200 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c231edb-6dcf-4414-9419-a27242efbbb8 None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:37.674886 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c231edb-6dcf-4414-9419-a27242efbbb8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:37.675191 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c231edb-6dcf-4414-9419-a27242efbbb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:37.676567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c231edb-6dcf-4414-9419-a27242efbbb8 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:37.685505 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 138/275] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 14:12:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:37.691463 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28975aa0-de56-4c74-b282-c67d10b29123 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:37.703631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:37.810645 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28975aa0-de56-4c74-b282-c67d10b29123 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:37.839347 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf47cc6-c605-4593-9a36-efce200058a5 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:37.839632 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf47cc6-c605-4593-9a36-efce200058a5 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:37.839904 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf47cc6-c605-4593-9a36-efce200058a5 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:37.841270 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 139/276] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:12:37.847913 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca1f8b-fed5-4fc3-a46d-a24f328242bd None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:37.849007 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca1f8b-fed5-4fc3-a46d-a24f328242bd None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:37.849487 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca1f8b-fed5-4fc3-a46d-a24f328242bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:37.869067 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fe87f3-719e-4b7c-8bac-6a2bc3ad8475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:37.870192 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe87f3-719e-4b7c-8bac-6a2bc3ad8475 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:37.870468 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe87f3-719e-4b7c-8bac-6a2bc3ad8475 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:37.870738 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe87f3-719e-4b7c-8bac-6a2bc3ad8475 None admin] PATH_INFO: `/v3/users/e9fd8bb0f2f44052a825e901b1c2c7bb` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:37.872258 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-c3ca1f8b-fed5-4fc3-a46d-a24f328242bd None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:37.873658 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-c3ca1f8b-fed5-4fc3-a46d-a24f328242bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:37.887030 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fe87f3-719e-4b7c-8bac-6a2bc3ad8475 None admin] RBAC: Authorizing `identity:get_user(user_id=e9fd8bb0f2f44052a825e901b1c2c7bb)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:37.888546 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fe87f3-719e-4b7c-8bac-6a2bc3ad8475 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:37.890231 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 138/277] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:12:37 2022] GET /identity/v3/users/e9fd8bb0f2f44052a825e901b1c2c7bb => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:37.891023 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fe87f3-719e-4b7c-8bac-6a2bc3ad8475 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:37.910866 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:37.951991 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 140/278] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:12:37.953113 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ca1f8b-fed5-4fc3-a46d-a24f328242bd None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:37.972507 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:38.018295 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fe87f3-719e-4b7c-8bac-6a2bc3ad8475 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:38.058552 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ca1f8b-fed5-4fc3-a46d-a24f328242bd None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:38.063080 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd515a-48b7-49fe-9bcd-b314d10c94ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:38.063725 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd515a-48b7-49fe-9bcd-b314d10c94ce None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:38.063998 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd515a-48b7-49fe-9bcd-b314d10c94ce None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:38.064265 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd515a-48b7-49fe-9bcd-b314d10c94ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:38.065356 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdd515a-48b7-49fe-9bcd-b314d10c94ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:38.066894 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdd515a-48b7-49fe-9bcd-b314d10c94ce None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:38.068832 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 139/279] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:12:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:38.074764 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd515a-48b7-49fe-9bcd-b314d10c94ce None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:38.098966 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b7734-93ee-463e-ad0c-0e0e8b3a7934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:38.099880 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b7734-93ee-463e-ad0c-0e0e8b3a7934 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:38.100358 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b7734-93ee-463e-ad0c-0e0e8b3a7934 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:38.100862 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b7734-93ee-463e-ad0c-0e0e8b3a7934 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:38.101859 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b7734-93ee-463e-ad0c-0e0e8b3a7934 None admin] RBAC: Adding query filter params (role.id=640af5182e6848498c16a19cddbfb738, user.id=e9fd8bb0f2f44052a825e901b1c2c7bb, scope.system=all) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:38.102380 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b7734-93ee-463e-ad0c-0e0e8b3a7934 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:38.104832 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b7734-93ee-463e-ad0c-0e0e8b3a7934 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:38.110589 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 141/280] 10.222.0.52 () {56 vars in 1308 bytes} [Sat Sep 24 14:12:37 2022] GET /identity/v3/role_assignments?user.id=e9fd8bb0f2f44052a825e901b1c2c7bb&scope.system=all&role.id=640af5182e6848498c16a19cddbfb738 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:38.187293 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd515a-48b7-49fe-9bcd-b314d10c94ce None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:38.223603 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b7734-93ee-463e-ad0c-0e0e8b3a7934 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:38.225862 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f4f43-6b22-49b0-bce5-306f99110f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:38.226489 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f4f43-6b22-49b0-bce5-306f99110f6c None admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:38.226765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f4f43-6b22-49b0-bce5-306f99110f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:38.227061 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f4f43-6b22-49b0-bce5-306f99110f6c None admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:38.227909 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f4f43-6b22-49b0-bce5-306f99110f6c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:38.230527 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f4f43-6b22-49b0-bce5-306f99110f6c None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:38.240212 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:38.259211 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 140/281] 10.222.0.52 () {60 vars in 1132 bytes} [Sat Sep 24 14:12:38 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:12:38.274791 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:38.364770 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b7734-93ee-463e-ad0c-0e0e8b3a7934 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:38.404788 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a586a59b-08e6-4439-85cf-47431f006fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:38.405816 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a586a59b-08e6-4439-85cf-47431f006fa2 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:38.406582 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a586a59b-08e6-4439-85cf-47431f006fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:38.407095 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a586a59b-08e6-4439-85cf-47431f006fa2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:38.419786 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087e029-29ac-4206-b170-8b2baf015e81 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:38.420123 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087e029-29ac-4206-b170-8b2baf015e81 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:38.420336 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087e029-29ac-4206-b170-8b2baf015e81 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:38.421044 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a586a59b-08e6-4439-85cf-47431f006fa2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:38.423250 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 141/282] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:12:38.426795 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-a586a59b-08e6-4439-85cf-47431f006fa2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 14:12:38.426795 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:38.426795 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:38.426795 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:38.426795 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:38.426795 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:38.426795 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:38.426795 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:38.426795 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:38.426795 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:38.426795 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:38.426795 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:38.426795 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:38.426795 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:38.426795 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:38.426795 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:38.427719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:38.427719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:38.427719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:38.427719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:38.427719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:38.427719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:38.427719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:38.427719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:38.427719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:38.427719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:38.427719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:38.427719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:38.427719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:38.427719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:38.427719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:38.427719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:38.428512 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:38.428512 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:38.428512 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:38.428512 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:38.428512 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:38.428512 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:38.428512 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:38.428512 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:38.428512 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:38.428512 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:38.428512 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 14:12:38.428512 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:12:38.429093 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526bde1-69d5-425d-82bb-7dec80199d73 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:38.429093 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526bde1-69d5-425d-82bb-7dec80199d73 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:38.429093 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526bde1-69d5-425d-82bb-7dec80199d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:38.431546 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 142/283] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:12:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:12:38.437691 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a586a59b-08e6-4439-85cf-47431f006fa2 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:38.453814 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-f526bde1-69d5-425d-82bb-7dec80199d73 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:38.455034 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-f526bde1-69d5-425d-82bb-7dec80199d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:38.547756 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 142/284] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:12:38.560743 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:38.570667 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a586a59b-08e6-4439-85cf-47431f006fa2 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:38.613995 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bebbb53-6b23-43ee-b656-f1a862a6e9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:38.615105 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebbb53-6b23-43ee-b656-f1a862a6e9ec None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:38.615614 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebbb53-6b23-43ee-b656-f1a862a6e9ec None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:38.615939 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebbb53-6b23-43ee-b656-f1a862a6e9ec None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:38.616630 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bebbb53-6b23-43ee-b656-f1a862a6e9ec None admin] RBAC: Adding query filter params (name=reader) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:38.616936 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bebbb53-6b23-43ee-b656-f1a862a6e9ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:38.618324 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bebbb53-6b23-43ee-b656-f1a862a6e9ec None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:38.629419 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 143/285] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 14:12:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:38.635543 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526bde1-69d5-425d-82bb-7dec80199d73 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:38.753492 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526bde1-69d5-425d-82bb-7dec80199d73 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:38.784281 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:38.800909 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e8b0d-3e45-4620-8efd-4be029be4d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:38.801682 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e8b0d-3e45-4620-8efd-4be029be4d13 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:38.802227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e8b0d-3e45-4620-8efd-4be029be4d13 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:38.802506 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e8b0d-3e45-4620-8efd-4be029be4d13 None admin] PATH_INFO: `/v3/users/fed403235e194e86adc513853e82c550` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:38.803749 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257e8b0d-3e45-4620-8efd-4be029be4d13 None admin] RBAC: Authorizing `identity:get_user(user_id=fed403235e194e86adc513853e82c550)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:38.805308 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257e8b0d-3e45-4620-8efd-4be029be4d13 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:38.807306 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 143/286] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:12:38 2022] GET /identity/v3/users/fed403235e194e86adc513853e82c550 => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:38.812510 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bebbb53-6b23-43ee-b656-f1a862a6e9ec None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:38.835333 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e8b0d-3e45-4620-8efd-4be029be4d13 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:38.974471 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e8b0d-3e45-4620-8efd-4be029be4d13 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:38.987082 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bebbb53-6b23-43ee-b656-f1a862a6e9ec None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:39.016292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40798aee-ab20-424d-bc20-e313a8bb2356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:39.017309 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40798aee-ab20-424d-bc20-e313a8bb2356 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:39.017587 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40798aee-ab20-424d-bc20-e313a8bb2356 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:39.017862 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40798aee-ab20-424d-bc20-e313a8bb2356 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:39.029426 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40798aee-ab20-424d-bc20-e313a8bb2356 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:39.035205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-40798aee-ab20-424d-bc20-e313a8bb2356 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:12:39.035205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:39.035205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:39.035205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:39.035205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:39.035205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:39.035205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:39.035205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:39.035205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:39.035205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:39.035205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:39.035205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:39.035205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:39.035205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:39.035205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:39.035205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:39.036046 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:39.036046 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:39.036046 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:39.036046 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:39.036046 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:39.036046 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:39.036046 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:39.036046 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:39.036046 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:39.036046 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:39.036046 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:39.036046 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:39.036046 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:39.036046 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:39.036046 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:39.036046 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:39.036804 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:39.036804 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:39.036804 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:39.036804 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:39.036804 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:39.036804 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:39.036804 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:39.036804 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:39.036804 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:39.036804 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:39.036804 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:12:39.036804 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:12:39.041642 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 144/287] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:12:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:12:39.043343 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40798aee-ab20-424d-bc20-e313a8bb2356 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:39.080094 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88406fed-e14c-4c72-bebd-779d8142ff9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:39.080756 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88406fed-e14c-4c72-bebd-779d8142ff9d None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:39.081038 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88406fed-e14c-4c72-bebd-779d8142ff9d None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:39.081317 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88406fed-e14c-4c72-bebd-779d8142ff9d None admin] PATH_INFO: `/v3/projects/c73b2a612ca646969294ac368768fcb5` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:39.082458 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88406fed-e14c-4c72-bebd-779d8142ff9d None admin] RBAC: Authorizing `identity:get_project(project_id=c73b2a612ca646969294ac368768fcb5)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:39.086916 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88406fed-e14c-4c72-bebd-779d8142ff9d None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:39.086916 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 144/288] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:12:38 2022] GET /identity/v3/projects/c73b2a612ca646969294ac368768fcb5 => generated 302 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:39.094238 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88406fed-e14c-4c72-bebd-779d8142ff9d None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:39.122975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:39.135913 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:39.189381 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40798aee-ab20-424d-bc20-e313a8bb2356 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:39.231475 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d26749-412d-4151-9fbd-0aee425b7d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:39.232669 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d26749-412d-4151-9fbd-0aee425b7d08 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:39.233190 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d26749-412d-4151-9fbd-0aee425b7d08 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:39.233655 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d26749-412d-4151-9fbd-0aee425b7d08 None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:39.234512 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d26749-412d-4151-9fbd-0aee425b7d08 None admin] RBAC: Adding query filter params (name=member) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:39.235042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d26749-412d-4151-9fbd-0aee425b7d08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:39.237566 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d26749-412d-4151-9fbd-0aee425b7d08 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:39.252456 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 145/289] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 14:12:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:39.253975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d26749-412d-4151-9fbd-0aee425b7d08 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:39.302634 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88406fed-e14c-4c72-bebd-779d8142ff9d None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:39.374782 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ed117-b9c1-4b4e-af35-a5167c616e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:39.375833 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ed117-b9c1-4b4e-af35-a5167c616e0a None admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:39.376414 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ed117-b9c1-4b4e-af35-a5167c616e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:39.376894 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ed117-b9c1-4b4e-af35-a5167c616e0a None admin] PATH_INFO: `/v3/projects/c73b2a612ca646969294ac368768fcb5/users/fed403235e194e86adc513853e82c550/roles/651d60b8bf604682a0609ba8ad8c3f1b` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:39.379698 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64ed117-b9c1-4b4e-af35-a5167c616e0a None admin] RBAC: Authorizing `identity:create_grant(project_id=c73b2a612ca646969294ac368768fcb5, user_id=fed403235e194e86adc513853e82c550, role_id=651d60b8bf604682a0609ba8ad8c3f1b)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:39.383170 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64ed117-b9c1-4b4e-af35-a5167c616e0a None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:39.409540 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 145/290] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 14:12:39 2022] PUT /identity/v3/projects/c73b2a612ca646969294ac368768fcb5/users/fed403235e194e86adc513853e82c550/roles/651d60b8bf604682a0609ba8ad8c3f1b => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:12:39.423313 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d26749-412d-4151-9fbd-0aee425b7d08 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:39.438999 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:39.466066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e479c19-f68b-445b-8d93-e5f42bc1c2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:39.467083 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e479c19-f68b-445b-8d93-e5f42bc1c2df None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:39.467561 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e479c19-f68b-445b-8d93-e5f42bc1c2df None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:39.468053 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e479c19-f68b-445b-8d93-e5f42bc1c2df None admin] PATH_INFO: `/v3/users/cb791a7f757b4646bf3647ea3069d7eb` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:39.469500 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e479c19-f68b-445b-8d93-e5f42bc1c2df None admin] RBAC: Authorizing `identity:get_user(user_id=cb791a7f757b4646bf3647ea3069d7eb)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:39.471283 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e479c19-f68b-445b-8d93-e5f42bc1c2df None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:39.473257 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 146/291] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:12:39 2022] GET /identity/v3/users/cb791a7f757b4646bf3647ea3069d7eb => generated 297 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:39.479444 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ed117-b9c1-4b4e-af35-a5167c616e0a None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:39.578678 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:39.601271 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ed117-b9c1-4b4e-af35-a5167c616e0a None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:39.639620 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e4cd7-45c4-48a4-b62f-21e1e0241bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:39.640596 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e4cd7-45c4-48a4-b62f-21e1e0241bdd None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:39.641069 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e4cd7-45c4-48a4-b62f-21e1e0241bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:39.641554 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e4cd7-45c4-48a4-b62f-21e1e0241bdd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:39.718905 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543e4cd7-45c4-48a4-b62f-21e1e0241bdd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:39.720692 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543e4cd7-45c4-48a4-b62f-21e1e0241bdd None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:39.805040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:39.833245 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 146/292] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:12:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:12:39.835474 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e479c19-f68b-445b-8d93-e5f42bc1c2df None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:39.955267 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e479c19-f68b-445b-8d93-e5f42bc1c2df None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:40.004295 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cc8bc4-4932-43ea-81a3-0dd967671560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:40.004979 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc8bc4-4932-43ea-81a3-0dd967671560 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:40.005266 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc8bc4-4932-43ea-81a3-0dd967671560 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:40.005540 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc8bc4-4932-43ea-81a3-0dd967671560 None admin] PATH_INFO: `/v3/projects/c73b2a612ca646969294ac368768fcb5` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:40.006668 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cc8bc4-4932-43ea-81a3-0dd967671560 None admin] RBAC: Authorizing `identity:get_project(project_id=c73b2a612ca646969294ac368768fcb5)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:40.008226 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cc8bc4-4932-43ea-81a3-0dd967671560 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:40.010204 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 147/293] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:12:39 2022] GET /identity/v3/projects/c73b2a612ca646969294ac368768fcb5 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:40.016071 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e4cd7-45c4-48a4-b62f-21e1e0241bdd None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:40.124339 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e4cd7-45c4-48a4-b62f-21e1e0241bdd None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:40.159677 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b385ff32-e666-40a5-87a5-addcb8bad79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:40.160663 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385ff32-e666-40a5-87a5-addcb8bad79e None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:40.161139 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385ff32-e666-40a5-87a5-addcb8bad79e None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:40.161608 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385ff32-e666-40a5-87a5-addcb8bad79e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:40.162557 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b385ff32-e666-40a5-87a5-addcb8bad79e None admin] RBAC: Adding query filter params (role.id=640af5182e6848498c16a19cddbfb738, scope.project.id=c73b2a612ca646969294ac368768fcb5, user.id=cb791a7f757b4646bf3647ea3069d7eb) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:40.163090 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b385ff32-e666-40a5-87a5-addcb8bad79e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:40.164827 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b385ff32-e666-40a5-87a5-addcb8bad79e None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:40.178902 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 147/294] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:12:40 2022] GET /identity/v3/role_assignments?user.id=cb791a7f757b4646bf3647ea3069d7eb&scope.project.id=c73b2a612ca646969294ac368768fcb5&role.id=640af5182e6848498c16a19cddbfb738 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:40.681138 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64104b1f-b1c8-4e3a-8ecf-1d133ee4be70 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:40.681575 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64104b1f-b1c8-4e3a-8ecf-1d133ee4be70 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:40.681769 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64104b1f-b1c8-4e3a-8ecf-1d133ee4be70 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:40.683459 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 148/295] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:40.689002 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418934a5-8c87-4f23-a8b6-6ab3ae17e79d None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:40.689294 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418934a5-8c87-4f23-a8b6-6ab3ae17e79d None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:40.689573 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418934a5-8c87-4f23-a8b6-6ab3ae17e79d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:40.711707 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-418934a5-8c87-4f23-a8b6-6ab3ae17e79d None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:40.713481 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-418934a5-8c87-4f23-a8b6-6ab3ae17e79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:40.762390 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:40.793810 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 148/296] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:41.008028 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32f0e4-a3c5-4b7d-a94e-c704f5490b9f None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:41.009059 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32f0e4-a3c5-4b7d-a94e-c704f5490b9f None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:41.010070 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32f0e4-a3c5-4b7d-a94e-c704f5490b9f None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:41.011920 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 149/297] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:41.017358 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fdf8bf-3851-4f13-b65f-0c6d4caeaa1d None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:41.018165 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fdf8bf-3851-4f13-b65f-0c6d4caeaa1d None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:41.018637 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fdf8bf-3851-4f13-b65f-0c6d4caeaa1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:41.038165 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-90fdf8bf-3851-4f13-b65f-0c6d4caeaa1d None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:41.039671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-90fdf8bf-3851-4f13-b65f-0c6d4caeaa1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:41.094582 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee32f0e4-a3c5-4b7d-a94e-c704f5490b9f None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:41.117693 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 149/298] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:41.149546 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:41.211319 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee32f0e4-a3c5-4b7d-a94e-c704f5490b9f None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:41.247868 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f145da8d-ce1e-4bf5-a529-5b956b1cae51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:41.248790 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145da8d-ce1e-4bf5-a529-5b956b1cae51 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:41.249322 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145da8d-ce1e-4bf5-a529-5b956b1cae51 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:41.249815 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145da8d-ce1e-4bf5-a529-5b956b1cae51 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:41.261099 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f145da8d-ce1e-4bf5-a529-5b956b1cae51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:41.266571 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-f145da8d-ce1e-4bf5-a529-5b956b1cae51 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:12:41.266571 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:41.266571 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:41.266571 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:41.266571 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:41.266571 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:41.266571 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:41.266571 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:41.266571 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:41.266571 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:41.266571 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:41.266571 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:41.266571 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:41.266571 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:41.266571 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:41.266571 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:41.267447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:41.267447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:41.267447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:41.267447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:41.267447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:41.267447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:41.267447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:41.267447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:41.267447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:41.267447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:41.267447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:41.267447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:41.267447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:41.267447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:41.267447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:41.267447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:41.268228 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:41.268228 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:41.268228 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:41.268228 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:41.268228 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:41.268228 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:41.268228 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:41.268228 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:41.268228 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:41.268228 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:41.268228 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:12:41.268228 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:12:41.270883 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 150/299] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:12:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:12:41.282975 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fdf8bf-3851-4f13-b65f-0c6d4caeaa1d None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:41.294135 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27afe9-f788-41f5-b0e7-0af6b9fd4b33 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:41.294415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27afe9-f788-41f5-b0e7-0af6b9fd4b33 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:41.294685 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27afe9-f788-41f5-b0e7-0af6b9fd4b33 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:41.296027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 151/300] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:12:41.300754 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e1c75-3ac8-463c-b75a-ff8a1f888101 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:41.301035 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e1c75-3ac8-463c-b75a-ff8a1f888101 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:41.301305 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e1c75-3ac8-463c-b75a-ff8a1f888101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:41.319881 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-e01e1c75-3ac8-463c-b75a-ff8a1f888101 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:41.321067 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-e01e1c75-3ac8-463c-b75a-ff8a1f888101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:41.392550 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fdf8bf-3851-4f13-b65f-0c6d4caeaa1d None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:41.395139 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 152/301] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:12:41.395649 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:41.404779 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e1c75-3ac8-463c-b75a-ff8a1f888101 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:41.432106 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e9ec5-5098-40fd-ab2b-6fc289e3a9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:41.433000 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e9ec5-5098-40fd-ab2b-6fc289e3a9ba None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:41.433285 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e9ec5-5098-40fd-ab2b-6fc289e3a9ba None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:41.433561 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e9ec5-5098-40fd-ab2b-6fc289e3a9ba None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:41.434332 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e9ec5-5098-40fd-ab2b-6fc289e3a9ba None admin] RBAC: Adding query filter params (name=member) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:41.434736 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e9ec5-5098-40fd-ab2b-6fc289e3a9ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:41.436147 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e9ec5-5098-40fd-ab2b-6fc289e3a9ba None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:41.445439 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 150/302] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 14:12:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:41.451423 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:41.453382 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e9ec5-5098-40fd-ab2b-6fc289e3a9ba None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:41.515305 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e1c75-3ac8-463c-b75a-ff8a1f888101 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:41.558338 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6fd2e5-7868-443c-a632-4bae1e589c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:41.559497 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6fd2e5-7868-443c-a632-4bae1e589c79 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:41.559839 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6fd2e5-7868-443c-a632-4bae1e589c79 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:41.560423 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6fd2e5-7868-443c-a632-4bae1e589c79 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:41.561753 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6fd2e5-7868-443c-a632-4bae1e589c79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:41.563876 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6fd2e5-7868-443c-a632-4bae1e589c79 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:41.566502 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 153/303] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:12:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:41.571752 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6fd2e5-7868-443c-a632-4bae1e589c79 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:41.574693 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e9ec5-5098-40fd-ab2b-6fc289e3a9ba None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:41.614290 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ff2214-7889-46de-b2f3-f59bba85ae4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:41.614960 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff2214-7889-46de-b2f3-f59bba85ae4b None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:41.615241 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff2214-7889-46de-b2f3-f59bba85ae4b None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:41.615509 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff2214-7889-46de-b2f3-f59bba85ae4b None admin] PATH_INFO: `/v3/users/e9fd8bb0f2f44052a825e901b1c2c7bb` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:41.617456 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ff2214-7889-46de-b2f3-f59bba85ae4b None admin] RBAC: Authorizing `identity:get_user(user_id=e9fd8bb0f2f44052a825e901b1c2c7bb)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:41.619117 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ff2214-7889-46de-b2f3-f59bba85ae4b None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:41.620708 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 151/304] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:12:41 2022] GET /identity/v3/users/e9fd8bb0f2f44052a825e901b1c2c7bb => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:41.626515 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ff2214-7889-46de-b2f3-f59bba85ae4b None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:41.665829 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:41.698011 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6fd2e5-7868-443c-a632-4bae1e589c79 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:41.705993 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:41.736055 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921ed46d-ae0f-4cd7-8030-543d7f37904a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:41.736670 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ed46d-ae0f-4cd7-8030-543d7f37904a None admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:41.736948 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ed46d-ae0f-4cd7-8030-543d7f37904a None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:41.737217 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ed46d-ae0f-4cd7-8030-543d7f37904a None admin] PATH_INFO: `/v3/groups` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:41.737487 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ff2214-7889-46de-b2f3-f59bba85ae4b None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:41.737845 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921ed46d-ae0f-4cd7-8030-543d7f37904a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:41.739336 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921ed46d-ae0f-4cd7-8030-543d7f37904a None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:41.756262 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 154/305] 10.222.0.52 () {60 vars in 1134 bytes} [Sat Sep 24 14:12:41 2022] POST /identity/v3/groups => generated 229 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:12:41.757085 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921ed46d-ae0f-4cd7-8030-543d7f37904a None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:41.791995 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f44aa3d-c5c8-44b8-868e-051dc85186d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:41.792870 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f44aa3d-c5c8-44b8-868e-051dc85186d3 None admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:41.793155 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f44aa3d-c5c8-44b8-868e-051dc85186d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:41.793445 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f44aa3d-c5c8-44b8-868e-051dc85186d3 None admin] PATH_INFO: `/v3/system/users/e9fd8bb0f2f44052a825e901b1c2c7bb/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:41.795115 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f44aa3d-c5c8-44b8-868e-051dc85186d3 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e9fd8bb0f2f44052a825e901b1c2c7bb, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:41.796692 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f44aa3d-c5c8-44b8-868e-051dc85186d3 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:41.810720 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 152/306] 10.222.0.52 () {58 vars in 1334 bytes} [Sat Sep 24 14:12:41 2022] PUT /identity/v3/system/users/e9fd8bb0f2f44052a825e901b1c2c7bb/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:12:41.912750 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921ed46d-ae0f-4cd7-8030-543d7f37904a None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:41.957219 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a80b05-064e-4732-bfcc-6ef7243bbaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:41.958261 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a80b05-064e-4732-bfcc-6ef7243bbaf8 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:41.958741 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a80b05-064e-4732-bfcc-6ef7243bbaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:41.959338 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a80b05-064e-4732-bfcc-6ef7243bbaf8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:41.972735 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a80b05-064e-4732-bfcc-6ef7243bbaf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:41.978496 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-f0a80b05-064e-4732-bfcc-6ef7243bbaf8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 14:12:41.978496 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:41.978496 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:41.978496 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:41.978496 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:41.978496 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:41.978496 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:41.978496 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:41.978496 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:41.978496 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:41.978496 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:41.978496 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:41.978496 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:41.978496 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:41.978496 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:41.978496 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:41.979462 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:41.979462 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:41.979462 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:41.979462 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:41.979462 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:41.979462 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:41.979462 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:41.979462 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:41.979462 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:41.979462 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:41.979462 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:41.979462 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:41.979462 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:41.979462 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:41.979462 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:41.979462 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:41.980220 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:41.980220 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:41.980220 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:41.980220 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:41.980220 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:41.980220 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:41.980220 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:41.980220 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:41.980220 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:41.980220 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:41.980220 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 14:12:41.980220 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:12:41.980220 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 155/307] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:12:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:12:41.985875 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f44aa3d-c5c8-44b8-868e-051dc85186d3 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:42.010895 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426eaa61-5cc9-4fb0-b778-9a17be74fe08 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:42.011404 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426eaa61-5cc9-4fb0-b778-9a17be74fe08 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:42.011484 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426eaa61-5cc9-4fb0-b778-9a17be74fe08 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:42.013409 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 156/308] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:42.020356 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c28fa0-17cf-4eaf-8c40-71b4748ab67d None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:42.020356 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c28fa0-17cf-4eaf-8c40-71b4748ab67d None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:42.020356 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c28fa0-17cf-4eaf-8c40-71b4748ab67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:42.039053 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-d4c28fa0-17cf-4eaf-8c40-71b4748ab67d None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:42.040327 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-d4c28fa0-17cf-4eaf-8c40-71b4748ab67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:42.068660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:42.106326 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:42.111440 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f44aa3d-c5c8-44b8-868e-051dc85186d3 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:42.136091 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 157/309] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:12:42.156217 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a1a57-b6fb-4121-9e5b-cbda01c45ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:42.157084 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a1a57-b6fb-4121-9e5b-cbda01c45ff2 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:42.157362 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a1a57-b6fb-4121-9e5b-cbda01c45ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:42.157630 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a1a57-b6fb-4121-9e5b-cbda01c45ff2 None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:42.158298 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789a1a57-b6fb-4121-9e5b-cbda01c45ff2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:42.158653 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789a1a57-b6fb-4121-9e5b-cbda01c45ff2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:42.160179 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789a1a57-b6fb-4121-9e5b-cbda01c45ff2 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:42.173934 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 153/310] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 14:12:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:42.175661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c28fa0-17cf-4eaf-8c40-71b4748ab67d None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:42.293788 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c28fa0-17cf-4eaf-8c40-71b4748ab67d None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:42.335523 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d241cb84-1ec6-44d2-8e09-5b2e20669ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:42.336494 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d241cb84-1ec6-44d2-8e09-5b2e20669ab7 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:42.336965 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d241cb84-1ec6-44d2-8e09-5b2e20669ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:42.337447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d241cb84-1ec6-44d2-8e09-5b2e20669ab7 None admin] PATH_INFO: `/v3/users/955db31c94f0481aa5a25a7eaa9b3d5e` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:42.351490 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d241cb84-1ec6-44d2-8e09-5b2e20669ab7 None admin] RBAC: Authorizing `identity:get_user(user_id=955db31c94f0481aa5a25a7eaa9b3d5e)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:42.353228 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d241cb84-1ec6-44d2-8e09-5b2e20669ab7 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:42.355590 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 158/311] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:12:42 2022] GET /identity/v3/users/955db31c94f0481aa5a25a7eaa9b3d5e => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:42.361922 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a1a57-b6fb-4121-9e5b-cbda01c45ff2 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:42.410182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d241cb84-1ec6-44d2-8e09-5b2e20669ab7 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:42.435176 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:42.474228 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a1a57-b6fb-4121-9e5b-cbda01c45ff2 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:42.513266 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e71d9-44ac-4157-af86-423780cd117e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:42.513996 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e71d9-44ac-4157-af86-423780cd117e None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:42.514346 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e71d9-44ac-4157-af86-423780cd117e None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:42.514779 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e71d9-44ac-4157-af86-423780cd117e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:42.515708 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0e71d9-44ac-4157-af86-423780cd117e None admin] RBAC: Adding query filter params (role.id=651d60b8bf604682a0609ba8ad8c3f1b, user.id=955db31c94f0481aa5a25a7eaa9b3d5e, scope.system=all) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:42.516093 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0e71d9-44ac-4157-af86-423780cd117e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:42.517801 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0e71d9-44ac-4157-af86-423780cd117e None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:42.523816 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 154/312] 10.222.0.52 () {56 vars in 1308 bytes} [Sat Sep 24 14:12:42 2022] GET /identity/v3/role_assignments?user.id=955db31c94f0481aa5a25a7eaa9b3d5e&scope.system=all&role.id=651d60b8bf604682a0609ba8ad8c3f1b => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:42.528563 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d241cb84-1ec6-44d2-8e09-5b2e20669ab7 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:42.575035 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9578ca3c-a67e-4190-9470-7f776134dc48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:42.576025 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578ca3c-a67e-4190-9470-7f776134dc48 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:42.576482 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578ca3c-a67e-4190-9470-7f776134dc48 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:42.576941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578ca3c-a67e-4190-9470-7f776134dc48 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:42.590099 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9578ca3c-a67e-4190-9470-7f776134dc48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:42.595515 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-9578ca3c-a67e-4190-9470-7f776134dc48 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 14:12:42.595515 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:42.595515 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:42.595515 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:42.595515 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:42.595515 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:42.595515 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:42.595515 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:42.595515 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:42.595515 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:42.595515 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:42.595515 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:42.595515 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:42.595515 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:42.595515 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:42.595515 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:42.596578 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:42.596578 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:42.596578 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:42.596578 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:42.596578 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:42.596578 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:42.596578 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:42.596578 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:42.596578 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:42.596578 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:42.596578 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:42.596578 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:42.596578 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:42.596578 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:42.596578 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:42.596578 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:42.597359 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:42.597359 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:42.597359 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:42.597359 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:42.597359 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:42.597359 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:42.597359 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:42.597359 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:42.597359 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:42.597359 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:42.597359 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 14:12:42.597359 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:12:42.599868 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 159/313] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:12:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:12:42.606881 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e71d9-44ac-4157-af86-423780cd117e None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:42.635663 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:42.724884 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e71d9-44ac-4157-af86-423780cd117e None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:42.762097 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43408930-0fa3-47e5-a493-5a9f7cfc6200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:42.762721 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43408930-0fa3-47e5-a493-5a9f7cfc6200 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:42.763030 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43408930-0fa3-47e5-a493-5a9f7cfc6200 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:42.763300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43408930-0fa3-47e5-a493-5a9f7cfc6200 None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:42.763973 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43408930-0fa3-47e5-a493-5a9f7cfc6200 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:42.764277 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43408930-0fa3-47e5-a493-5a9f7cfc6200 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:42.765659 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43408930-0fa3-47e5-a493-5a9f7cfc6200 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:42.775133 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 155/314] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 14:12:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:42.781041 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9578ca3c-a67e-4190-9470-7f776134dc48 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:42.867873 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:42.895081 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9578ca3c-a67e-4190-9470-7f776134dc48 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:42.932326 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f984680-7621-45e0-a712-247a3e623e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:42.933259 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f984680-7621-45e0-a712-247a3e623e02 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:42.933687 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f984680-7621-45e0-a712-247a3e623e02 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:42.934101 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f984680-7621-45e0-a712-247a3e623e02 None admin] PATH_INFO: `/v3/users/fed403235e194e86adc513853e82c550` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:42.935484 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f984680-7621-45e0-a712-247a3e623e02 None admin] RBAC: Authorizing `identity:get_user(user_id=fed403235e194e86adc513853e82c550)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:42.937134 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f984680-7621-45e0-a712-247a3e623e02 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:42.939202 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 160/315] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:12:42 2022] GET /identity/v3/users/fed403235e194e86adc513853e82c550 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:42.944975 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43408930-0fa3-47e5-a493-5a9f7cfc6200 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:43.036378 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:43.052097 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43408930-0fa3-47e5-a493-5a9f7cfc6200 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:43.089184 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2308e255-bd81-429e-b947-63d0ce32c4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:43.089800 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308e255-bd81-429e-b947-63d0ce32c4cd None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:43.090254 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308e255-bd81-429e-b947-63d0ce32c4cd None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:43.090531 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308e255-bd81-429e-b947-63d0ce32c4cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:43.163491 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2308e255-bd81-429e-b947-63d0ce32c4cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:43.164967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2308e255-bd81-429e-b947-63d0ce32c4cd None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:43.253192 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:43.269535 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 156/316] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:12:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:12:43.275293 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f984680-7621-45e0-a712-247a3e623e02 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:43.385630 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f984680-7621-45e0-a712-247a3e623e02 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:43.421095 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02570dd0-3a91-412a-91d6-0062979be53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:43.421928 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02570dd0-3a91-412a-91d6-0062979be53c None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:43.422343 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02570dd0-3a91-412a-91d6-0062979be53c None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:43.422746 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02570dd0-3a91-412a-91d6-0062979be53c None admin] PATH_INFO: `/v3/projects/c73b2a612ca646969294ac368768fcb5` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:43.424009 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02570dd0-3a91-412a-91d6-0062979be53c None admin] RBAC: Authorizing `identity:get_project(project_id=c73b2a612ca646969294ac368768fcb5)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:43.425596 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02570dd0-3a91-412a-91d6-0062979be53c None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:43.427394 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 161/317] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:12:43 2022] GET /identity/v3/projects/c73b2a612ca646969294ac368768fcb5 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:43.433891 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2308e255-bd81-429e-b947-63d0ce32c4cd None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:43.538281 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2308e255-bd81-429e-b947-63d0ce32c4cd None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:43.573384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c43708-08e1-4911-bc84-cb2809b34966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:43.574217 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c43708-08e1-4911-bc84-cb2809b34966 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:43.574657 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c43708-08e1-4911-bc84-cb2809b34966 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:43.575091 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c43708-08e1-4911-bc84-cb2809b34966 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:43.575945 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c43708-08e1-4911-bc84-cb2809b34966 None admin] RBAC: Adding query filter params (role.id=651d60b8bf604682a0609ba8ad8c3f1b, scope.project.id=c73b2a612ca646969294ac368768fcb5, user.id=fed403235e194e86adc513853e82c550) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:43.576385 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c43708-08e1-4911-bc84-cb2809b34966 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:43.577944 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c43708-08e1-4911-bc84-cb2809b34966 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:43.584055 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 157/318] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:12:43 2022] GET /identity/v3/role_assignments?user.id=fed403235e194e86adc513853e82c550&scope.project.id=c73b2a612ca646969294ac368768fcb5&role.id=651d60b8bf604682a0609ba8ad8c3f1b => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:44.065596 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9ba9b-9c5d-4f39-aaba-76ddb67a0516 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:44.065994 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9ba9b-9c5d-4f39-aaba-76ddb67a0516 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:44.066167 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9ba9b-9c5d-4f39-aaba-76ddb67a0516 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:44.067752 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 162/319] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:44.073465 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f18b0ed-8106-4f6f-a9d4-781329234796 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:44.073773 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f18b0ed-8106-4f6f-a9d4-781329234796 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:44.074027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f18b0ed-8106-4f6f-a9d4-781329234796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:44.093281 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-8f18b0ed-8106-4f6f-a9d4-781329234796 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:44.094411 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-8f18b0ed-8106-4f6f-a9d4-781329234796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:44.141341 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:44.163477 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 158/320] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:44.172789 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b65f7-b013-4707-8313-1fe7ad7b6d54 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:44.173075 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b65f7-b013-4707-8313-1fe7ad7b6d54 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:44.173350 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b65f7-b013-4707-8313-1fe7ad7b6d54 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:44.174804 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 163/321] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:44.179808 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ec216-6f38-4115-80e8-c70d67fe9be1 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:44.180234 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ec216-6f38-4115-80e8-c70d67fe9be1 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:44.180386 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ec216-6f38-4115-80e8-c70d67fe9be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:44.198288 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-b33ec216-6f38-4115-80e8-c70d67fe9be1 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:44.199485 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-b33ec216-6f38-4115-80e8-c70d67fe9be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:44.266284 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 159/322] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:44.427984 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b65f7-b013-4707-8313-1fe7ad7b6d54 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:44.440595 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:44.531781 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b65f7-b013-4707-8313-1fe7ad7b6d54 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:44.540169 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33ec216-6f38-4115-80e8-c70d67fe9be1 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:44.567136 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e620c-d7ca-44c8-a089-bcf5e4002ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:44.567773 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e620c-d7ca-44c8-a089-bcf5e4002ecd None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:44.568047 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e620c-d7ca-44c8-a089-bcf5e4002ecd None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:44.568317 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e620c-d7ca-44c8-a089-bcf5e4002ecd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:44.578833 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e620c-d7ca-44c8-a089-bcf5e4002ecd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:44.583597 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-f15e620c-d7ca-44c8-a089-bcf5e4002ecd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:12:44.583597 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:44.583597 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:44.583597 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:44.583597 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:44.583597 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:44.583597 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:44.583597 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:44.583597 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:44.583597 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:44.583597 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:44.583597 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:44.583597 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:44.583597 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:44.583597 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:44.583597 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:44.584479 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:44.584479 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:44.584479 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:44.584479 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:44.584479 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:44.584479 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:44.584479 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:44.584479 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:44.584479 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:44.584479 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:44.584479 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:44.584479 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:44.584479 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:44.584479 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:44.584479 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:44.584479 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:44.585252 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:44.585252 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:44.585252 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:44.585252 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:44.585252 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:44.585252 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:44.585252 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:44.585252 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:44.585252 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:44.585252 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:44.585252 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:12:44.585252 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:12:44.585252 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 164/323] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:12:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:12:44.592112 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e620c-d7ca-44c8-a089-bcf5e4002ecd None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:44.659941 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33ec216-6f38-4115-80e8-c70d67fe9be1 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:44.666372 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:44.673769 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:44.699194 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18cc4e-0fbe-437d-87fe-609f660f1914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:44.699528 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e620c-d7ca-44c8-a089-bcf5e4002ecd None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:44.699753 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18cc4e-0fbe-437d-87fe-609f660f1914 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:44.700025 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18cc4e-0fbe-437d-87fe-609f660f1914 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:44.700288 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18cc4e-0fbe-437d-87fe-609f660f1914 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:44.701467 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18cc4e-0fbe-437d-87fe-609f660f1914 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:44.702987 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18cc4e-0fbe-437d-87fe-609f660f1914 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:44.704713 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 160/324] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:12:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:44.709995 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18cc4e-0fbe-437d-87fe-609f660f1914 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:44.738549 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a439342-db43-44a0-a1d9-210c61538c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:44.739300 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a439342-db43-44a0-a1d9-210c61538c7d None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:44.739580 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a439342-db43-44a0-a1d9-210c61538c7d None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:44.739848 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a439342-db43-44a0-a1d9-210c61538c7d None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:44.740510 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a439342-db43-44a0-a1d9-210c61538c7d None admin] RBAC: Adding query filter params (name=member) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:44.740810 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a439342-db43-44a0-a1d9-210c61538c7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:44.742178 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a439342-db43-44a0-a1d9-210c61538c7d None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:44.751077 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 165/325] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 14:12:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:44.752224 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6835bd05-1ff5-4bae-b0e7-c3a4af11c4c5 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:44.752506 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6835bd05-1ff5-4bae-b0e7-c3a4af11c4c5 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:44.752778 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6835bd05-1ff5-4bae-b0e7-c3a4af11c4c5 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:44.753968 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 166/326] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:44.756734 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6835bd05-1ff5-4bae-b0e7-c3a4af11c4c5 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:44.812516 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18cc4e-0fbe-437d-87fe-609f660f1914 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:44.849660 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce0857-54d1-4971-9e1a-16879cbb2cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:44.850266 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce0857-54d1-4971-9e1a-16879cbb2cf3 None admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:44.850539 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce0857-54d1-4971-9e1a-16879cbb2cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:44.850803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce0857-54d1-4971-9e1a-16879cbb2cf3 None admin] PATH_INFO: `/v3/groups` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:44.851451 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce0857-54d1-4971-9e1a-16879cbb2cf3 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:44.852854 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce0857-54d1-4971-9e1a-16879cbb2cf3 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:44.857429 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6835bd05-1ff5-4bae-b0e7-c3a4af11c4c5 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:44.868973 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 161/327] 10.222.0.52 () {60 vars in 1134 bytes} [Sat Sep 24 14:12:44 2022] POST /identity/v3/groups => generated 226 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:12:44.883047 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5f382-adfd-4604-86a2-8c8e4f84bfeb None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:44.883047 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5f382-adfd-4604-86a2-8c8e4f84bfeb None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:44.883047 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5f382-adfd-4604-86a2-8c8e4f84bfeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:44.891445 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:44.892436 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-bbe5f382-adfd-4604-86a2-8c8e4f84bfeb None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:44.893910 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-bbe5f382-adfd-4604-86a2-8c8e4f84bfeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:44.898884 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b13f7d-c5aa-4173-a3dc-1110a6334f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:44.899545 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b13f7d-c5aa-4173-a3dc-1110a6334f1f None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:44.899837 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b13f7d-c5aa-4173-a3dc-1110a6334f1f None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:44.900097 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b13f7d-c5aa-4173-a3dc-1110a6334f1f None admin] PATH_INFO: `/v3/users/e9fd8bb0f2f44052a825e901b1c2c7bb` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:44.901825 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b13f7d-c5aa-4173-a3dc-1110a6334f1f None admin] RBAC: Authorizing `identity:get_user(user_id=e9fd8bb0f2f44052a825e901b1c2c7bb)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:44.903396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b13f7d-c5aa-4173-a3dc-1110a6334f1f None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:44.905095 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 167/328] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:12:44 2022] GET /identity/v3/users/e9fd8bb0f2f44052a825e901b1c2c7bb => generated 293 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:44.911505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b13f7d-c5aa-4173-a3dc-1110a6334f1f None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:44.935236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:44.984074 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 162/329] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:45.023766 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b13f7d-c5aa-4173-a3dc-1110a6334f1f None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:45.063486 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cef2609-7cb5-4289-80ce-a97d717f5727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:45.064482 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef2609-7cb5-4289-80ce-a97d717f5727 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:45.064957 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef2609-7cb5-4289-80ce-a97d717f5727 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:45.065296 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef2609-7cb5-4289-80ce-a97d717f5727 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:45.066058 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cef2609-7cb5-4289-80ce-a97d717f5727 None admin] RBAC: Adding query filter params (role.id=640af5182e6848498c16a19cddbfb738, user.id=e9fd8bb0f2f44052a825e901b1c2c7bb, scope.system=all) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:45.066360 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cef2609-7cb5-4289-80ce-a97d717f5727 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:45.067878 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cef2609-7cb5-4289-80ce-a97d717f5727 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:45.081053 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 168/330] 10.222.0.52 () {56 vars in 1308 bytes} [Sat Sep 24 14:12:44 2022] GET /identity/v3/role_assignments?user.id=e9fd8bb0f2f44052a825e901b1c2c7bb&scope.system=all&role.id=640af5182e6848498c16a19cddbfb738 => generated 510 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:45.252902 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5f382-adfd-4604-86a2-8c8e4f84bfeb None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:45.360690 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5f382-adfd-4604-86a2-8c8e4f84bfeb None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:45.395282 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c448a-0edf-4af4-b3ee-e32c73870b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:45.395916 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c448a-0edf-4af4-b3ee-e32c73870b68 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:45.396190 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c448a-0edf-4af4-b3ee-e32c73870b68 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:45.396457 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c448a-0edf-4af4-b3ee-e32c73870b68 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:45.407305 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405c448a-0edf-4af4-b3ee-e32c73870b68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:45.412123 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-405c448a-0edf-4af4-b3ee-e32c73870b68 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 14:12:45.412123 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:45.412123 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:45.412123 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:45.412123 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:45.412123 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:45.412123 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:45.412123 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:45.412123 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:45.412123 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:45.412123 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:45.412123 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:45.412123 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:45.412123 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:45.412123 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:45.412123 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:45.413003 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:45.413003 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:45.413003 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:45.413003 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:45.413003 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:45.413003 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:45.413003 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:45.413003 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:45.413003 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:45.413003 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:45.413003 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:45.413003 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:45.413003 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:45.413003 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:45.413003 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:45.413003 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:45.413774 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:45.413774 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:45.413774 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:45.413774 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:45.413774 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:45.413774 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:45.413774 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:45.413774 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:45.413774 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:45.413774 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:45.413774 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 14:12:45.413774 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:12:45.413774 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 163/331] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:12:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:12:45.419528 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cef2609-7cb5-4289-80ce-a97d717f5727 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:45.488791 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:45.535530 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cef2609-7cb5-4289-80ce-a97d717f5727 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:45.571865 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab4eba-15a8-416c-8766-66b3b22c9da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:45.573093 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab4eba-15a8-416c-8766-66b3b22c9da6 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:45.573367 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab4eba-15a8-416c-8766-66b3b22c9da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:45.573633 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab4eba-15a8-416c-8766-66b3b22c9da6 None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:45.574298 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab4eba-15a8-416c-8766-66b3b22c9da6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:45.574595 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab4eba-15a8-416c-8766-66b3b22c9da6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:45.576063 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab4eba-15a8-416c-8766-66b3b22c9da6 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:45.585056 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 169/332] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 14:12:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:45.590289 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c448a-0edf-4af4-b3ee-e32c73870b68 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:45.622741 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:45.717631 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c448a-0edf-4af4-b3ee-e32c73870b68 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:45.754584 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bb69c-19c2-4cd7-bc9f-5f19190e946e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:45.755316 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bb69c-19c2-4cd7-bc9f-5f19190e946e None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:45.755592 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bb69c-19c2-4cd7-bc9f-5f19190e946e None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:45.755863 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bb69c-19c2-4cd7-bc9f-5f19190e946e None admin] PATH_INFO: `/v3/users/955db31c94f0481aa5a25a7eaa9b3d5e` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:45.757044 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03bb69c-19c2-4cd7-bc9f-5f19190e946e None admin] RBAC: Authorizing `identity:get_user(user_id=955db31c94f0481aa5a25a7eaa9b3d5e)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:45.758557 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03bb69c-19c2-4cd7-bc9f-5f19190e946e None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:45.761047 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 164/333] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:12:45 2022] GET /identity/v3/users/955db31c94f0481aa5a25a7eaa9b3d5e => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:45.766893 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab4eba-15a8-416c-8766-66b3b22c9da6 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:45.899827 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab4eba-15a8-416c-8766-66b3b22c9da6 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:45.927487 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:45.946884 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb05a8-3aa7-490e-b090-6550352e1c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:45.947899 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb05a8-3aa7-490e-b090-6550352e1c86 None admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:45.948340 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb05a8-3aa7-490e-b090-6550352e1c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:45.948735 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb05a8-3aa7-490e-b090-6550352e1c86 None admin] PATH_INFO: `/v3/system/users/955db31c94f0481aa5a25a7eaa9b3d5e/roles/651d60b8bf604682a0609ba8ad8c3f1b` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:45.950167 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cb05a8-3aa7-490e-b090-6550352e1c86 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=955db31c94f0481aa5a25a7eaa9b3d5e, role_id=651d60b8bf604682a0609ba8ad8c3f1b)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:45.951639 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cb05a8-3aa7-490e-b090-6550352e1c86 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:45.968555 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 170/334] 10.222.0.52 () {58 vars in 1334 bytes} [Sat Sep 24 14:12:45 2022] PUT /identity/v3/system/users/955db31c94f0481aa5a25a7eaa9b3d5e/roles/651d60b8bf604682a0609ba8ad8c3f1b => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:12:47.319975 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ffe66-6495-4075-91de-c23e17d1562b None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:47.320588 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ffe66-6495-4075-91de-c23e17d1562b None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:47.320588 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ffe66-6495-4075-91de-c23e17d1562b None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:47.322252 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 165/335] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:47.332829 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ccf181-7f06-4848-80cc-cf7310db2a98 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:47.333713 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ccf181-7f06-4848-80cc-cf7310db2a98 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:47.334187 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ccf181-7f06-4848-80cc-cf7310db2a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:47.359832 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-77ccf181-7f06-4848-80cc-cf7310db2a98 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:47.361538 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-77ccf181-7f06-4848-80cc-cf7310db2a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:47.451483 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 171/336] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:47.748417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af98dae-d670-4a3c-9d03-5528446f735f None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:47.748700 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af98dae-d670-4a3c-9d03-5528446f735f None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:47.749162 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af98dae-d670-4a3c-9d03-5528446f735f None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:47.751023 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 166/337] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:47.757694 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab353b91-694c-49a4-aa10-417dc16ad574 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:47.757978 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab353b91-694c-49a4-aa10-417dc16ad574 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:47.758252 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab353b91-694c-49a4-aa10-417dc16ad574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:47.782377 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-ab353b91-694c-49a4-aa10-417dc16ad574 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:47.783642 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-ab353b91-694c-49a4-aa10-417dc16ad574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:47.881179 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 172/338] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:47.892101 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af98dae-d670-4a3c-9d03-5528446f735f None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:48.027151 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:48.034034 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af98dae-d670-4a3c-9d03-5528446f735f None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:48.064740 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fbd568-470a-43b6-9b25-81d18641abcb None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:48.065284 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fbd568-470a-43b6-9b25-81d18641abcb None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:48.065702 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fbd568-470a-43b6-9b25-81d18641abcb None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:48.067212 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 173/339] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:12:48.072656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c4433-bba2-4f49-8da1-2cb544d302d1 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:48.073104 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c4433-bba2-4f49-8da1-2cb544d302d1 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:48.073518 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c4433-bba2-4f49-8da1-2cb544d302d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:48.078850 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07302991-2a14-4143-a53f-ca00d8bbf959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:48.078850 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07302991-2a14-4143-a53f-ca00d8bbf959 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:48.078850 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07302991-2a14-4143-a53f-ca00d8bbf959 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:48.078850 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07302991-2a14-4143-a53f-ca00d8bbf959 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:48.098017 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07302991-2a14-4143-a53f-ca00d8bbf959 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:48.102964 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-07302991-2a14-4143-a53f-ca00d8bbf959 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:12:48.102964 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:48.102964 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:48.102964 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:48.102964 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:48.102964 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:48.102964 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:48.102964 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:48.102964 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:48.102964 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:48.102964 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:48.102964 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:48.102964 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:48.102964 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:48.102964 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:48.102964 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:48.104418 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:48.104418 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:48.104418 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:48.104418 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:48.104418 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:48.104418 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:48.104418 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:48.104418 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:48.104418 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:48.104418 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:48.104418 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:48.104418 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:48.104418 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:48.104418 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:48.104418 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:48.104418 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:48.105211 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:48.105211 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:48.105211 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:48.105211 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:48.105211 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:48.105211 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:48.105211 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:48.105211 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:48.105211 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:48.105211 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:48.105211 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:12:48.105211 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:12:48.105211 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 167/340] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:12:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:12:48.110915 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07302991-2a14-4143-a53f-ca00d8bbf959 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:48.112078 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-f70c4433-bba2-4f49-8da1-2cb544d302d1 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:48.113530 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-f70c4433-bba2-4f49-8da1-2cb544d302d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:48.185368 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:48.244761 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 174/341] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:12:48.285927 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07302991-2a14-4143-a53f-ca00d8bbf959 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:48.305593 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c4433-bba2-4f49-8da1-2cb544d302d1 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:48.330986 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef6982c-be29-4d7b-8bab-6cfb7c69788f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:48.331668 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6982c-be29-4d7b-8bab-6cfb7c69788f None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:48.331864 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6982c-be29-4d7b-8bab-6cfb7c69788f None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:48.332133 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6982c-be29-4d7b-8bab-6cfb7c69788f None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:48.332802 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef6982c-be29-4d7b-8bab-6cfb7c69788f None admin] RBAC: Adding query filter params (name=member) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:48.333096 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef6982c-be29-4d7b-8bab-6cfb7c69788f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:48.334467 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef6982c-be29-4d7b-8bab-6cfb7c69788f None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:48.347728 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 168/342] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 14:12:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:48.355277 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef6982c-be29-4d7b-8bab-6cfb7c69788f None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:48.404213 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:48.533733 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c4433-bba2-4f49-8da1-2cb544d302d1 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:48.553248 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef6982c-be29-4d7b-8bab-6cfb7c69788f None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:48.592665 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ad27b-4930-43df-a17e-1c5e83c304d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:48.593540 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ad27b-4930-43df-a17e-1c5e83c304d2 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:48.593956 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ad27b-4930-43df-a17e-1c5e83c304d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:48.594362 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ad27b-4930-43df-a17e-1c5e83c304d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:48.596019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a49caf-f1d0-4e84-868d-ad86d4258d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:48.596662 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a49caf-f1d0-4e84-868d-ad86d4258d35 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:48.596938 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a49caf-f1d0-4e84-868d-ad86d4258d35 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:48.597205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a49caf-f1d0-4e84-868d-ad86d4258d35 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:48.621135 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a49caf-f1d0-4e84-868d-ad86d4258d35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:48.626082 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-47a49caf-f1d0-4e84-868d-ad86d4258d35 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:12:48.626082 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:48.626082 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:48.626082 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:48.626082 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:48.626082 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:48.626082 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:48.626082 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:48.626082 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:48.626082 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:48.626082 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:48.626082 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:48.626082 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:48.626082 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:48.626082 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:48.626082 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:48.627098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:48.627098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:48.627098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:48.627098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:48.627098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:48.627098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:48.627098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:48.627098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:48.627098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:48.627098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:48.627098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:48.627098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:48.627098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:48.627098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:48.627098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:48.627098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:48.627949 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:48.627949 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:48.627949 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:48.627949 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:48.627949 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:48.627949 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:48.627949 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:48.627949 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:48.627949 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:48.627949 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:48.627949 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:12:48.627949 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:12:48.631318 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 175/343] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:12:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:12:48.639751 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a49caf-f1d0-4e84-868d-ad86d4258d35 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:48.679802 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01ad27b-4930-43df-a17e-1c5e83c304d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:48.681274 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01ad27b-4930-43df-a17e-1c5e83c304d2 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:48.707185 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:48.720767 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:48.826841 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a49caf-f1d0-4e84-868d-ad86d4258d35 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:48.876518 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 169/344] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:12:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:12:48.877388 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ad27b-4930-43df-a17e-1c5e83c304d2 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:48.898763 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e380c3e-7dc7-4f55-b321-d898c18f0dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:48.899404 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e380c3e-7dc7-4f55-b321-d898c18f0dfd None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:48.899682 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e380c3e-7dc7-4f55-b321-d898c18f0dfd None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:48.899950 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e380c3e-7dc7-4f55-b321-d898c18f0dfd None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:48.900667 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e380c3e-7dc7-4f55-b321-d898c18f0dfd None admin] RBAC: Adding query filter params (name=member) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:48.900965 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e380c3e-7dc7-4f55-b321-d898c18f0dfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:48.902432 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e380c3e-7dc7-4f55-b321-d898c18f0dfd None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:48.920440 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 176/345] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 14:12:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:48.921090 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e380c3e-7dc7-4f55-b321-d898c18f0dfd None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:49.013415 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ad27b-4930-43df-a17e-1c5e83c304d2 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:49.048154 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:49.052894 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c029a-a017-473b-b1b3-5520139c26c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:49.053522 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c029a-a017-473b-b1b3-5520139c26c3 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:49.053793 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c029a-a017-473b-b1b3-5520139c26c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:49.054058 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c029a-a017-473b-b1b3-5520139c26c3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:49.071749 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80c029a-a017-473b-b1b3-5520139c26c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:49.077169 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-f80c029a-a017-473b-b1b3-5520139c26c3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:12:49.077169 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:49.077169 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:49.077169 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:49.077169 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:49.077169 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:49.077169 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:49.077169 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:49.077169 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:49.077169 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:49.077169 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:49.077169 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:49.077169 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:49.077169 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:49.077169 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:49.077169 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:49.078027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:49.078027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:49.078027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:49.078027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:49.078027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:49.078027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:49.078027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:49.078027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:49.078027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:49.078027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:49.078027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:49.078027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:49.078027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:49.078027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:49.078027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:49.078027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:49.078917 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:49.078917 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:49.078917 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:49.078917 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:49.078917 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:49.078917 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:49.078917 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:49.078917 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:49.078917 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:49.078917 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:49.078917 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:12:49.078917 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:12:49.083212 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 170/346] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 14:12:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:12:49.084031 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c029a-a017-473b-b1b3-5520139c26c3 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:49.143617 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e380c3e-7dc7-4f55-b321-d898c18f0dfd None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:49.147117 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:49.211534 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87df681c-d435-4125-ae52-347aa7cc64b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:49.212196 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df681c-d435-4125-ae52-347aa7cc64b8 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:49.212481 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df681c-d435-4125-ae52-347aa7cc64b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:49.213441 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df681c-d435-4125-ae52-347aa7cc64b8 None admin] PATH_INFO: `/v3/projects/a7eea203d96a471cb590c418bb060d23` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:49.214632 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87df681c-d435-4125-ae52-347aa7cc64b8 None admin] RBAC: Authorizing `identity:get_project(project_id=a7eea203d96a471cb590c418bb060d23)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:49.216118 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87df681c-d435-4125-ae52-347aa7cc64b8 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:49.218054 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 177/347] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:12:48 2022] GET /identity/v3/projects/a7eea203d96a471cb590c418bb060d23 => generated 298 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:49.218843 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87df681c-d435-4125-ae52-347aa7cc64b8 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:49.226773 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c029a-a017-473b-b1b3-5520139c26c3 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:49.261881 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49deefb-eff8-44a2-8f97-db07e19b2afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:49.262492 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49deefb-eff8-44a2-8f97-db07e19b2afc None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:49.262771 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49deefb-eff8-44a2-8f97-db07e19b2afc None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:49.263070 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49deefb-eff8-44a2-8f97-db07e19b2afc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:49.313951 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:49.342132 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49deefb-eff8-44a2-8f97-db07e19b2afc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:49.343600 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49deefb-eff8-44a2-8f97-db07e19b2afc None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:49.396054 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87df681c-d435-4125-ae52-347aa7cc64b8 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:49.452207 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 171/348] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:12:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:12:49.453421 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49deefb-eff8-44a2-8f97-db07e19b2afc None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:49.461768 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86810ae1-3130-48c2-b388-560c6dbc2ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:49.462407 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86810ae1-3130-48c2-b388-560c6dbc2ebf None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:49.462726 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86810ae1-3130-48c2-b388-560c6dbc2ebf None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:49.463062 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86810ae1-3130-48c2-b388-560c6dbc2ebf None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:49.463745 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86810ae1-3130-48c2-b388-560c6dbc2ebf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:49.464210 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86810ae1-3130-48c2-b388-560c6dbc2ebf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:49.465766 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86810ae1-3130-48c2-b388-560c6dbc2ebf None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:49.485377 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 178/349] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:12:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:49.486521 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86810ae1-3130-48c2-b388-560c6dbc2ebf None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:49.522407 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:49.538329 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:49.561848 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49deefb-eff8-44a2-8f97-db07e19b2afc None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:49.601294 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d4249-78fb-4917-b404-2c5c050f74d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:49.601929 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d4249-78fb-4917-b404-2c5c050f74d3 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:49.602203 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d4249-78fb-4917-b404-2c5c050f74d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:49.602867 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d4249-78fb-4917-b404-2c5c050f74d3 None admin] PATH_INFO: `/v3/groups/9a8e9df9724d4fd88cc9fe6515aca1f4` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:49.609434 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d4249-78fb-4917-b404-2c5c050f74d3 None admin] RBAC: Authorizing `identity:get_group(group_id=9a8e9df9724d4fd88cc9fe6515aca1f4)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:49.610985 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d4249-78fb-4917-b404-2c5c050f74d3 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:49.612624 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 172/350] 10.222.0.52 () {56 vars in 1180 bytes} [Sat Sep 24 14:12:49 2022] GET /identity/v3/groups/9a8e9df9724d4fd88cc9fe6515aca1f4 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:49.613965 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261dd986-1568-45ab-8f70-da620a65ffa7 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:49.614248 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261dd986-1568-45ab-8f70-da620a65ffa7 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:49.614516 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261dd986-1568-45ab-8f70-da620a65ffa7 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:49.617039 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86810ae1-3130-48c2-b388-560c6dbc2ebf None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:49.620832 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 173/351] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:49.621459 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261dd986-1568-45ab-8f70-da620a65ffa7 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:49.668841 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c359a6-dd45-436b-b737-1e252e12784c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:49.669884 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c359a6-dd45-436b-b737-1e252e12784c None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:49.669884 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c359a6-dd45-436b-b737-1e252e12784c None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:49.670042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c359a6-dd45-436b-b737-1e252e12784c None admin] PATH_INFO: `/v3/projects/c73b2a612ca646969294ac368768fcb5` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:49.671615 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c359a6-dd45-436b-b737-1e252e12784c None admin] RBAC: Authorizing `identity:get_project(project_id=c73b2a612ca646969294ac368768fcb5)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:49.673071 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c359a6-dd45-436b-b737-1e252e12784c None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:49.674898 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 179/352] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:12:49 2022] GET /identity/v3/projects/c73b2a612ca646969294ac368768fcb5 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:49.678593 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f021281-bbc7-4baa-bb9a-abdc0472566f None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:49.679384 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f021281-bbc7-4baa-bb9a-abdc0472566f None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:49.679384 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f021281-bbc7-4baa-bb9a-abdc0472566f None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:49.680609 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 180/353] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:49.681180 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f021281-bbc7-4baa-bb9a-abdc0472566f None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:49.755527 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261dd986-1568-45ab-8f70-da620a65ffa7 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:49.797019 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae22577e-6bdc-4f42-acd5-03e82591eccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:49.797644 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22577e-6bdc-4f42-acd5-03e82591eccb None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:49.797927 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22577e-6bdc-4f42-acd5-03e82591eccb None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:49.798203 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22577e-6bdc-4f42-acd5-03e82591eccb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:49.799945 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:49.810005 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f021281-bbc7-4baa-bb9a-abdc0472566f None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:49.823063 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:49.862186 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d376e-8517-432b-8152-e37aee64cac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:49.862790 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d376e-8517-432b-8152-e37aee64cac0 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:49.863118 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d376e-8517-432b-8152-e37aee64cac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:49.863392 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d376e-8517-432b-8152-e37aee64cac0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:49.864508 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4d376e-8517-432b-8152-e37aee64cac0 None admin] RBAC: Adding query filter params (group.id=9a8e9df9724d4fd88cc9fe6515aca1f4, role.id=640af5182e6848498c16a19cddbfb738, scope.project.id=a7eea203d96a471cb590c418bb060d23) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:49.864821 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4d376e-8517-432b-8152-e37aee64cac0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:49.866633 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4d376e-8517-432b-8152-e37aee64cac0 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:49.873400 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 181/354] 10.222.0.52 () {56 vars in 1376 bytes} [Sat Sep 24 14:12:49 2022] GET /identity/v3/role_assignments?group.id=9a8e9df9724d4fd88cc9fe6515aca1f4&scope.project.id=a7eea203d96a471cb590c418bb060d23&role.id=640af5182e6848498c16a19cddbfb738 => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:49.875405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc69ddc-95b0-461d-88ec-5f4a1031c82e None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:49.875792 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc69ddc-95b0-461d-88ec-5f4a1031c82e None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:49.876168 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc69ddc-95b0-461d-88ec-5f4a1031c82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:49.899453 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae22577e-6bdc-4f42-acd5-03e82591eccb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:49.901721 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-7bc69ddc-95b0-461d-88ec-5f4a1031c82e None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:49.902197 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae22577e-6bdc-4f42-acd5-03e82591eccb None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:49.903176 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-7bc69ddc-95b0-461d-88ec-5f4a1031c82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:50.006610 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 182/355] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:50.007786 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc69ddc-95b0-461d-88ec-5f4a1031c82e None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:50.058148 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 174/356] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:12:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:12:50.060267 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31da974-e44c-4248-9ddc-29b2286ef08c None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:50.060660 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31da974-e44c-4248-9ddc-29b2286ef08c None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:50.061041 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31da974-e44c-4248-9ddc-29b2286ef08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:50.095797 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-f31da974-e44c-4248-9ddc-29b2286ef08c None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:50.096542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-f31da974-e44c-4248-9ddc-29b2286ef08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:50.142739 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:50.153386 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:50.157576 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc69ddc-95b0-461d-88ec-5f4a1031c82e None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:50.199644 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 175/357] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:50.200507 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31da974-e44c-4248-9ddc-29b2286ef08c None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:50.201469 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf3371c-1773-4490-a5fd-22ae501550ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:50.202258 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3371c-1773-4490-a5fd-22ae501550ee None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:50.202636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3371c-1773-4490-a5fd-22ae501550ee None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:50.203107 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3371c-1773-4490-a5fd-22ae501550ee None admin] PATH_INFO: `/v3/groups/9a8e9df9724d4fd88cc9fe6515aca1f4` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:50.204422 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf3371c-1773-4490-a5fd-22ae501550ee None admin] RBAC: Authorizing `identity:get_group(group_id=9a8e9df9724d4fd88cc9fe6515aca1f4)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:50.205974 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf3371c-1773-4490-a5fd-22ae501550ee None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:50.207892 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 183/358] 10.222.0.52 () {56 vars in 1180 bytes} [Sat Sep 24 14:12:50 2022] GET /identity/v3/groups/9a8e9df9724d4fd88cc9fe6515aca1f4 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:50.209036 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d87c7-9df1-4b49-9cdb-6a2e08f022e3 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:50.209419 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d87c7-9df1-4b49-9cdb-6a2e08f022e3 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:50.209794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d87c7-9df1-4b49-9cdb-6a2e08f022e3 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:50.211137 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 184/359] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:50.215144 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d87c7-9df1-4b49-9cdb-6a2e08f022e3 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:50.383732 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31da974-e44c-4248-9ddc-29b2286ef08c None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:50.387105 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d87c7-9df1-4b49-9cdb-6a2e08f022e3 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:50.421902 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eddb705-ad88-4750-9c96-6d42cd2cf0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:50.423087 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eddb705-ad88-4750-9c96-6d42cd2cf0f5 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:50.423644 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eddb705-ad88-4750-9c96-6d42cd2cf0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:50.424141 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eddb705-ad88-4750-9c96-6d42cd2cf0f5 None admin] PATH_INFO: `/v3/projects/a7eea203d96a471cb590c418bb060d23` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:50.425331 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c6015-448c-4d98-ae7f-babbe51d085c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:50.425820 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eddb705-ad88-4750-9c96-6d42cd2cf0f5 None admin] RBAC: Authorizing `identity:get_project(project_id=a7eea203d96a471cb590c418bb060d23)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:50.425924 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c6015-448c-4d98-ae7f-babbe51d085c None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:50.426205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c6015-448c-4d98-ae7f-babbe51d085c None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:50.426477 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c6015-448c-4d98-ae7f-babbe51d085c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:50.427237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c6015-448c-4d98-ae7f-babbe51d085c None admin] RBAC: Adding query filter params (group.id=9a8e9df9724d4fd88cc9fe6515aca1f4, role.id=640af5182e6848498c16a19cddbfb738, scope.project.id=c73b2a612ca646969294ac368768fcb5) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:50.427546 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c6015-448c-4d98-ae7f-babbe51d085c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:50.428443 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eddb705-ad88-4750-9c96-6d42cd2cf0f5 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:50.429027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c6015-448c-4d98-ae7f-babbe51d085c None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:50.430525 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 176/360] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:12:50 2022] GET /identity/v3/projects/a7eea203d96a471cb590c418bb060d23 => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:50.432474 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e4294-7ba6-4561-932c-5ebeefbefaa9 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:50.432969 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e4294-7ba6-4561-932c-5ebeefbefaa9 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:50.433454 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e4294-7ba6-4561-932c-5ebeefbefaa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:50.440844 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 185/361] 10.222.0.52 () {56 vars in 1376 bytes} [Sat Sep 24 14:12:50 2022] GET /identity/v3/role_assignments?group.id=9a8e9df9724d4fd88cc9fe6515aca1f4&scope.project.id=c73b2a612ca646969294ac368768fcb5&role.id=640af5182e6848498c16a19cddbfb738 => generated 260 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:50.441897 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c6015-448c-4d98-ae7f-babbe51d085c None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:50.455280 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-c58e4294-7ba6-4561-932c-5ebeefbefaa9 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:50.456817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-c58e4294-7ba6-4561-932c-5ebeefbefaa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:50.472080 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:50.498314 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:50.545994 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 177/362] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:50.551939 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58e4294-7ba6-4561-932c-5ebeefbefaa9 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:50.593666 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c6015-448c-4d98-ae7f-babbe51d085c None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:50.642485 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24598410-ca77-46de-97ad-52846d1950ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:50.643255 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24598410-ca77-46de-97ad-52846d1950ae None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:50.643535 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24598410-ca77-46de-97ad-52846d1950ae None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:50.643809 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24598410-ca77-46de-97ad-52846d1950ae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:50.655837 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24598410-ca77-46de-97ad-52846d1950ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:50.662142 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-24598410-ca77-46de-97ad-52846d1950ae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:12:50.662142 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:50.662142 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:50.662142 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:50.662142 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:50.662142 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:50.662142 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:50.662142 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:50.662142 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:50.662142 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:50.662142 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:50.662142 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:50.662142 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:50.662142 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:50.662142 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:50.662142 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:50.663354 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:50.663354 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:50.663354 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:50.663354 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:50.663354 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:50.663354 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:50.663354 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:50.663354 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:50.663354 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:50.663354 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:50.663354 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:50.663354 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:50.663354 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:50.663354 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:50.663354 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:50.663354 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:50.664376 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:50.664376 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:50.664376 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:50.664376 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:50.664376 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:50.664376 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:50.664376 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:50.664376 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:50.664376 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:50.664376 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:50.664376 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:12:50.664376 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:12:50.666993 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 186/363] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 14:12:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:12:50.668007 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24598410-ca77-46de-97ad-52846d1950ae None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:50.710617 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58e4294-7ba6-4561-932c-5ebeefbefaa9 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:50.749842 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d90c9-dbb5-4b6f-8213-0c6a3f34284c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:50.750488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d90c9-dbb5-4b6f-8213-0c6a3f34284c None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:50.750974 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d90c9-dbb5-4b6f-8213-0c6a3f34284c None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:50.751267 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d90c9-dbb5-4b6f-8213-0c6a3f34284c None admin] PATH_INFO: `/v3/groups/9a8e9df9724d4fd88cc9fe6515aca1f4` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:50.752368 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767d90c9-dbb5-4b6f-8213-0c6a3f34284c None admin] RBAC: Authorizing `identity:get_group(group_id=9a8e9df9724d4fd88cc9fe6515aca1f4)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:50.753798 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767d90c9-dbb5-4b6f-8213-0c6a3f34284c None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:50.755998 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 178/364] 10.222.0.52 () {56 vars in 1180 bytes} [Sat Sep 24 14:12:50 2022] GET /identity/v3/groups/9a8e9df9724d4fd88cc9fe6515aca1f4 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:50.756930 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d90c9-dbb5-4b6f-8213-0c6a3f34284c None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:50.762739 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:50.783877 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24598410-ca77-46de-97ad-52846d1950ae None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:50.809756 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:50.828096 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6a133-0141-405e-8b9a-0c52a9eff331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:50.829025 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6a133-0141-405e-8b9a-0c52a9eff331 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:50.829396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6a133-0141-405e-8b9a-0c52a9eff331 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:50.829799 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6a133-0141-405e-8b9a-0c52a9eff331 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:50.843053 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e6a133-0141-405e-8b9a-0c52a9eff331 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:50.849656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-50e6a133-0141-405e-8b9a-0c52a9eff331 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 14:12:50.849656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:50.849656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:50.849656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:50.849656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:50.849656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:50.849656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:50.849656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:50.849656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:50.849656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:50.849656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:50.849656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:50.849656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:50.849656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:50.849656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:50.849656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:50.850652 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:50.850652 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:50.850652 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:50.850652 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:50.850652 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:50.850652 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:50.850652 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:50.850652 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:50.850652 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:50.850652 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:50.850652 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:50.850652 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:50.850652 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:50.850652 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:50.850652 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:50.850652 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:50.851487 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:50.851487 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:50.851487 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:50.851487 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:50.851487 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:50.851487 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:50.851487 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:50.851487 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:50.851487 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:50.851487 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:50.851487 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 14:12:50.851487 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:12:50.852089 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 187/365] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:12:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:12:50.852853 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6a133-0141-405e-8b9a-0c52a9eff331 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:50.881475 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d90c9-dbb5-4b6f-8213-0c6a3f34284c None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:50.919253 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554c9ea-e89d-452a-8dc9-4874595c8d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:50.920194 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554c9ea-e89d-452a-8dc9-4874595c8d2c None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:50.920656 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554c9ea-e89d-452a-8dc9-4874595c8d2c None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:50.921108 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554c9ea-e89d-452a-8dc9-4874595c8d2c None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:50.921965 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d554c9ea-e89d-452a-8dc9-4874595c8d2c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:50.922449 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d554c9ea-e89d-452a-8dc9-4874595c8d2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:50.924067 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d554c9ea-e89d-452a-8dc9-4874595c8d2c None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:50.933472 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 179/366] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:12:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:50.934625 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554c9ea-e89d-452a-8dc9-4874595c8d2c None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:50.972804 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6a133-0141-405e-8b9a-0c52a9eff331 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:51.013393 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470639a7-283a-449b-8ea4-bbe11b4a1029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:51.014004 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470639a7-283a-449b-8ea4-bbe11b4a1029 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:51.014336 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470639a7-283a-449b-8ea4-bbe11b4a1029 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:51.014608 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470639a7-283a-449b-8ea4-bbe11b4a1029 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:51.015472 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470639a7-283a-449b-8ea4-bbe11b4a1029 None admin] RBAC: Adding query filter params (group.id=9a8e9df9724d4fd88cc9fe6515aca1f4, role.id=25257d00ad2640c2a57d257b0279155f, scope.project.id=a7eea203d96a471cb590c418bb060d23) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:51.015777 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470639a7-283a-449b-8ea4-bbe11b4a1029 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:51.017552 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470639a7-283a-449b-8ea4-bbe11b4a1029 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:51.023832 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 188/367] 10.222.0.52 () {56 vars in 1376 bytes} [Sat Sep 24 14:12:50 2022] GET /identity/v3/role_assignments?group.id=9a8e9df9724d4fd88cc9fe6515aca1f4&scope.project.id=a7eea203d96a471cb590c418bb060d23&role.id=25257d00ad2640c2a57d257b0279155f => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:51.024642 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470639a7-283a-449b-8ea4-bbe11b4a1029 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:51.042848 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:51.071684 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:51.077929 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554c9ea-e89d-452a-8dc9-4874595c8d2c None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:51.139676 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722ac1fa-9ecc-421a-a456-9ae5b0ae64b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:51.140718 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ac1fa-9ecc-421a-a456-9ae5b0ae64b7 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:51.141224 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ac1fa-9ecc-421a-a456-9ae5b0ae64b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:51.141702 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ac1fa-9ecc-421a-a456-9ae5b0ae64b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:51.142608 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722ac1fa-9ecc-421a-a456-9ae5b0ae64b7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:51.143149 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722ac1fa-9ecc-421a-a456-9ae5b0ae64b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:51.144776 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722ac1fa-9ecc-421a-a456-9ae5b0ae64b7 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:51.147255 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470639a7-283a-449b-8ea4-bbe11b4a1029 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:51.155358 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 180/368] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 14:12:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:51.156530 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722ac1fa-9ecc-421a-a456-9ae5b0ae64b7 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:51.210352 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83466e5f-70f5-4fc9-bf18-2cf3b266021b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:51.211051 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83466e5f-70f5-4fc9-bf18-2cf3b266021b None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:51.211334 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83466e5f-70f5-4fc9-bf18-2cf3b266021b None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:51.211868 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83466e5f-70f5-4fc9-bf18-2cf3b266021b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:51.239947 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83466e5f-70f5-4fc9-bf18-2cf3b266021b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:51.248933 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-83466e5f-70f5-4fc9-bf18-2cf3b266021b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:12:51.248933 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:51.248933 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:51.248933 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:51.248933 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:51.248933 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:51.248933 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:51.248933 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:51.248933 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:51.248933 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:51.248933 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:51.248933 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:51.248933 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:51.248933 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:51.248933 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:51.248933 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:51.249890 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:51.249890 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:51.249890 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:51.249890 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:51.249890 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:51.249890 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:51.249890 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:51.249890 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:51.249890 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:51.249890 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:51.249890 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:51.249890 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:51.249890 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:51.249890 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:51.249890 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:51.249890 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:51.250674 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:51.250674 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:51.250674 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:51.250674 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:51.250674 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:51.250674 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:51.250674 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:51.250674 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:51.250674 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:51.250674 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:51.250674 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:12:51.250674 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:12:51.250674 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 189/369] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 14:12:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:12:51.252419 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83466e5f-70f5-4fc9-bf18-2cf3b266021b None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:51.341638 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722ac1fa-9ecc-421a-a456-9ae5b0ae64b7 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:51.381709 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73425c4e-5ef3-42fc-a007-b9ff47eede90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:51.382706 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73425c4e-5ef3-42fc-a007-b9ff47eede90 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:51.383226 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73425c4e-5ef3-42fc-a007-b9ff47eede90 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:51.383711 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73425c4e-5ef3-42fc-a007-b9ff47eede90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:51.387010 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:51.435940 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:51.454093 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83466e5f-70f5-4fc9-bf18-2cf3b266021b None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:51.513296 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73425c4e-5ef3-42fc-a007-b9ff47eede90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:51.513971 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6972ede-bdda-48d6-9770-fddb01b0bf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:51.514700 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73425c4e-5ef3-42fc-a007-b9ff47eede90 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:51.515130 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6972ede-bdda-48d6-9770-fddb01b0bf9f None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:51.515612 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6972ede-bdda-48d6-9770-fddb01b0bf9f None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:51.516103 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6972ede-bdda-48d6-9770-fddb01b0bf9f None admin] PATH_INFO: `/v3/users/955db31c94f0481aa5a25a7eaa9b3d5e` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:51.517688 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6972ede-bdda-48d6-9770-fddb01b0bf9f None admin] RBAC: Authorizing `identity:get_user(user_id=955db31c94f0481aa5a25a7eaa9b3d5e)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:51.519433 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6972ede-bdda-48d6-9770-fddb01b0bf9f None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:51.521669 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 190/370] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:12:51 2022] GET /identity/v3/users/955db31c94f0481aa5a25a7eaa9b3d5e => generated 293 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:51.523644 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6972ede-bdda-48d6-9770-fddb01b0bf9f None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:51.694710 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 181/371] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:12:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:12:51.699912 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73425c4e-5ef3-42fc-a007-b9ff47eede90 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:51.709678 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6972ede-bdda-48d6-9770-fddb01b0bf9f None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:51.775381 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:51.778233 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:51.787851 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca16a43-5e11-4b06-b388-0021761cd243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:51.788465 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca16a43-5e11-4b06-b388-0021761cd243 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:51.788741 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca16a43-5e11-4b06-b388-0021761cd243 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:51.789013 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca16a43-5e11-4b06-b388-0021761cd243 None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:51.789669 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca16a43-5e11-4b06-b388-0021761cd243 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:51.789971 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca16a43-5e11-4b06-b388-0021761cd243 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:51.791424 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca16a43-5e11-4b06-b388-0021761cd243 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:51.810647 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 191/372] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:12:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:51.811380 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca16a43-5e11-4b06-b388-0021761cd243 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:51.828098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73425c4e-5ef3-42fc-a007-b9ff47eede90 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:51.866582 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b082546-2e0c-4b8e-a82d-06d2b46dd5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:51.867479 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b082546-2e0c-4b8e-a82d-06d2b46dd5e2 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:51.867901 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b082546-2e0c-4b8e-a82d-06d2b46dd5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:51.868330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b082546-2e0c-4b8e-a82d-06d2b46dd5e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:51.869742 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b082546-2e0c-4b8e-a82d-06d2b46dd5e2 None admin] RBAC: Adding query filter params (role.id=651d60b8bf604682a0609ba8ad8c3f1b, user.id=955db31c94f0481aa5a25a7eaa9b3d5e, scope.system=all) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:51.870217 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b082546-2e0c-4b8e-a82d-06d2b46dd5e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:51.871992 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b082546-2e0c-4b8e-a82d-06d2b46dd5e2 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:51.884751 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 182/373] 10.222.0.52 () {56 vars in 1308 bytes} [Sat Sep 24 14:12:51 2022] GET /identity/v3/role_assignments?user.id=955db31c94f0481aa5a25a7eaa9b3d5e&scope.system=all&role.id=651d60b8bf604682a0609ba8ad8c3f1b => generated 510 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:51.886333 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b082546-2e0c-4b8e-a82d-06d2b46dd5e2 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:52.001719 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca16a43-5e11-4b06-b388-0021761cd243 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:52.039932 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b082546-2e0c-4b8e-a82d-06d2b46dd5e2 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:52.073104 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:52.084093 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2866259e-6132-436a-ac0b-b5e09be125bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:52.084995 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866259e-6132-436a-ac0b-b5e09be125bc None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:52.085445 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866259e-6132-436a-ac0b-b5e09be125bc None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:52.085881 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866259e-6132-436a-ac0b-b5e09be125bc None admin] PATH_INFO: `/v3/projects/c73b2a612ca646969294ac368768fcb5` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:52.088889 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2866259e-6132-436a-ac0b-b5e09be125bc None admin] RBAC: Authorizing `identity:get_project(project_id=c73b2a612ca646969294ac368768fcb5)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:52.090587 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2866259e-6132-436a-ac0b-b5e09be125bc None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:52.092554 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 192/374] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:12:51 2022] GET /identity/v3/projects/c73b2a612ca646969294ac368768fcb5 => generated 302 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:52.098928 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2866259e-6132-436a-ac0b-b5e09be125bc None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:52.110951 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc947f45-3aea-410b-8533-e5c708bdd1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:52.111962 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc947f45-3aea-410b-8533-e5c708bdd1d8 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:52.112551 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc947f45-3aea-410b-8533-e5c708bdd1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:52.112975 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc947f45-3aea-410b-8533-e5c708bdd1d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:52.205907 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc947f45-3aea-410b-8533-e5c708bdd1d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:52.210843 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc947f45-3aea-410b-8533-e5c708bdd1d8 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:52.284461 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:52.330219 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 183/375] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:12:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:12:52.335788 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc947f45-3aea-410b-8533-e5c708bdd1d8 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:52.375284 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:52.420902 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2866259e-6132-436a-ac0b-b5e09be125bc None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:52.453290 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc947f45-3aea-410b-8533-e5c708bdd1d8 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:52.498325 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d900d28-72d1-474b-947b-c41df4052592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:52.499187 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d900d28-72d1-474b-947b-c41df4052592 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:52.499550 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d900d28-72d1-474b-947b-c41df4052592 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:52.499926 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d900d28-72d1-474b-947b-c41df4052592 None admin] PATH_INFO: `/v3/projects/2f8134960517435ea813cbf2d41e25d7` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:52.504598 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d900d28-72d1-474b-947b-c41df4052592 None admin] RBAC: Authorizing `identity:get_project(project_id=2f8134960517435ea813cbf2d41e25d7)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:52.504598 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d900d28-72d1-474b-947b-c41df4052592 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:52.504598 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 184/376] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:12:52 2022] GET /identity/v3/projects/2f8134960517435ea813cbf2d41e25d7 => generated 344 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:52.505691 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3784be5d-8241-411d-ac21-0d3def7908c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:52.506517 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784be5d-8241-411d-ac21-0d3def7908c8 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:52.506801 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784be5d-8241-411d-ac21-0d3def7908c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:52.507099 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784be5d-8241-411d-ac21-0d3def7908c8 None admin] PATH_INFO: `/v3/groups/9a8e9df9724d4fd88cc9fe6515aca1f4` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:52.508390 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3784be5d-8241-411d-ac21-0d3def7908c8 None admin] RBAC: Authorizing `identity:get_group(group_id=9a8e9df9724d4fd88cc9fe6515aca1f4)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:52.509708 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d900d28-72d1-474b-947b-c41df4052592 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:52.509981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3784be5d-8241-411d-ac21-0d3def7908c8 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:52.511580 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 193/377] 10.222.0.52 () {56 vars in 1180 bytes} [Sat Sep 24 14:12:52 2022] GET /identity/v3/groups/9a8e9df9724d4fd88cc9fe6515aca1f4 => generated 226 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:52.517341 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3784be5d-8241-411d-ac21-0d3def7908c8 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:52.616003 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:52.618065 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d900d28-72d1-474b-947b-c41df4052592 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:52.660590 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4efcc4-3f5c-4956-be86-dad3990e1c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:52.661606 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4efcc4-3f5c-4956-be86-dad3990e1c02 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:52.662065 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4efcc4-3f5c-4956-be86-dad3990e1c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:52.662527 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4efcc4-3f5c-4956-be86-dad3990e1c02 None admin] PATH_INFO: `/v3/groups/e5a29351527643de8cac05b4be913b03` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:52.668671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4efcc4-3f5c-4956-be86-dad3990e1c02 None admin] RBAC: Authorizing `identity:get_group(group_id=e5a29351527643de8cac05b4be913b03)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:52.671182 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4efcc4-3f5c-4956-be86-dad3990e1c02 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:52.673474 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 185/378] 10.222.0.52 () {56 vars in 1180 bytes} [Sat Sep 24 14:12:52 2022] GET /identity/v3/groups/e5a29351527643de8cac05b4be913b03 => generated 229 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:52.679841 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4efcc4-3f5c-4956-be86-dad3990e1c02 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:52.725716 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3784be5d-8241-411d-ac21-0d3def7908c8 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:52.763816 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24551d47-628d-4ddf-84db-36febe1a9ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:52.764454 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24551d47-628d-4ddf-84db-36febe1a9ca9 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:52.764729 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24551d47-628d-4ddf-84db-36febe1a9ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:52.764994 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24551d47-628d-4ddf-84db-36febe1a9ca9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:52.765741 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24551d47-628d-4ddf-84db-36febe1a9ca9 None admin] RBAC: Adding query filter params (group.id=9a8e9df9724d4fd88cc9fe6515aca1f4, role.id=25257d00ad2640c2a57d257b0279155f, scope.project.id=c73b2a612ca646969294ac368768fcb5) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:52.766101 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24551d47-628d-4ddf-84db-36febe1a9ca9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:52.767739 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24551d47-628d-4ddf-84db-36febe1a9ca9 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:52.771189 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:52.774569 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 194/379] 10.222.0.52 () {56 vars in 1376 bytes} [Sat Sep 24 14:12:52 2022] GET /identity/v3/role_assignments?group.id=9a8e9df9724d4fd88cc9fe6515aca1f4&scope.project.id=c73b2a612ca646969294ac368768fcb5&role.id=25257d00ad2640c2a57d257b0279155f => generated 260 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:52.904028 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b93224-9e7f-4184-8fe5-d23f25ce3449 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:52.904305 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b93224-9e7f-4184-8fe5-d23f25ce3449 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:52.904745 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b93224-9e7f-4184-8fe5-d23f25ce3449 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:52.905888 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4efcc4-3f5c-4956-be86-dad3990e1c02 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:52.906580 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 195/380] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:12:52.915357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6246f5c-efbf-4621-b191-ce8730d4fd3b None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:52.915682 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6246f5c-efbf-4621-b191-ce8730d4fd3b None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:52.915989 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6246f5c-efbf-4621-b191-ce8730d4fd3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:52.950385 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-b6246f5c-efbf-4621-b191-ce8730d4fd3b None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:52.951659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-b6246f5c-efbf-4621-b191-ce8730d4fd3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:52.959026 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e383110-75e9-4bb8-9914-7446cc08d535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:52.959716 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e383110-75e9-4bb8-9914-7446cc08d535 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:52.959996 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e383110-75e9-4bb8-9914-7446cc08d535 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:52.960269 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e383110-75e9-4bb8-9914-7446cc08d535 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:52.961080 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e383110-75e9-4bb8-9914-7446cc08d535 None admin] RBAC: Adding query filter params (group.id=e5a29351527643de8cac05b4be913b03, role.id=d00b78c32dbf4cc7925a55bb621d8fcf, scope.project.id=2f8134960517435ea813cbf2d41e25d7) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:52.961395 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e383110-75e9-4bb8-9914-7446cc08d535 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:52.963462 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e383110-75e9-4bb8-9914-7446cc08d535 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:52.977930 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 186/381] 10.222.0.52 () {56 vars in 1376 bytes} [Sat Sep 24 14:12:52 2022] GET /identity/v3/role_assignments?group.id=e5a29351527643de8cac05b4be913b03&scope.project.id=2f8134960517435ea813cbf2d41e25d7&role.id=d00b78c32dbf4cc7925a55bb621d8fcf => generated 260 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:53.058894 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 196/382] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:53.432427 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e383110-75e9-4bb8-9914-7446cc08d535 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:53.454996 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:53.559967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e383110-75e9-4bb8-9914-7446cc08d535 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:53.606459 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a1e13-8454-4395-b434-2e596521bc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:53.607141 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a1e13-8454-4395-b434-2e596521bc65 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:53.607420 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a1e13-8454-4395-b434-2e596521bc65 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:53.607694 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a1e13-8454-4395-b434-2e596521bc65 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:53.624086 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a1e13-8454-4395-b434-2e596521bc65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:53.629416 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-058a1e13-8454-4395-b434-2e596521bc65 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:12:53.629416 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:53.629416 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:53.629416 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:53.629416 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:53.629416 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:53.629416 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:53.629416 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:53.629416 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:53.629416 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:53.629416 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:53.629416 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:53.629416 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:53.629416 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:53.629416 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:53.629416 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:53.630250 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:53.630250 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:53.630250 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:53.630250 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:53.630250 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:53.630250 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:53.630250 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:53.630250 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:53.630250 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:53.630250 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:53.630250 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:53.630250 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:53.630250 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:53.630250 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:53.630250 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:53.630250 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:53.631044 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:53.631044 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:53.631044 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:53.631044 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:53.631044 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:53.631044 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:53.631044 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:53.631044 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:53.631044 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:53.631044 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:53.631044 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:12:53.631044 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:12:53.631044 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 187/383] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:12:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:12:53.638640 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6246f5c-efbf-4621-b191-ce8730d4fd3b None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:53.640937 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41adf619-6ec9-456b-ab54-a097274248ff None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:53.641226 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41adf619-6ec9-456b-ab54-a097274248ff None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:53.641499 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41adf619-6ec9-456b-ab54-a097274248ff None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:53.643090 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 188/384] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:53.654123 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc82831-2f08-447d-bb0e-d18c5c340772 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:53.654405 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc82831-2f08-447d-bb0e-d18c5c340772 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:53.654678 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc82831-2f08-447d-bb0e-d18c5c340772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:53.688066 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-1cc82831-2f08-447d-bb0e-d18c5c340772 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:53.689193 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-1cc82831-2f08-447d-bb0e-d18c5c340772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:53.799515 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 189/385] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:53.855469 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6246f5c-efbf-4621-b191-ce8730d4fd3b None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:53.885198 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:53.898670 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065d7734-0b51-494d-8149-7b251438b8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:53.899334 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d7734-0b51-494d-8149-7b251438b8b9 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:53.899613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d7734-0b51-494d-8149-7b251438b8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:53.899886 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d7734-0b51-494d-8149-7b251438b8b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:53.900604 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065d7734-0b51-494d-8149-7b251438b8b9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:53.900905 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065d7734-0b51-494d-8149-7b251438b8b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:53.902307 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065d7734-0b51-494d-8149-7b251438b8b9 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:53.912788 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 197/386] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 14:12:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:53.918569 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc82831-2f08-447d-bb0e-d18c5c340772 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:53.934730 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:54.046943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682d1f6-5063-49f5-981d-6c6baa73546a None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:54.047123 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682d1f6-5063-49f5-981d-6c6baa73546a None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:54.047338 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682d1f6-5063-49f5-981d-6c6baa73546a None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:54.048870 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 198/387] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:54.053350 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc82831-2f08-447d-bb0e-d18c5c340772 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:54.057849 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91087df1-bb75-4635-af70-a198154365e1 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:54.058130 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91087df1-bb75-4635-af70-a198154365e1 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:54.058401 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91087df1-bb75-4635-af70-a198154365e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:54.081206 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-91087df1-bb75-4635-af70-a198154365e1 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:54.082409 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-91087df1-bb75-4635-af70-a198154365e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:54.112307 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8778e10-44bb-4d90-840e-44de173daf58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:54.113311 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8778e10-44bb-4d90-840e-44de173daf58 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:54.114270 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8778e10-44bb-4d90-840e-44de173daf58 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:54.114753 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8778e10-44bb-4d90-840e-44de173daf58 None admin] PATH_INFO: `/v3/groups/9a8e9df9724d4fd88cc9fe6515aca1f4` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:54.116418 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8778e10-44bb-4d90-840e-44de173daf58 None admin] RBAC: Authorizing `identity:get_group(group_id=9a8e9df9724d4fd88cc9fe6515aca1f4)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:54.118082 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8778e10-44bb-4d90-840e-44de173daf58 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:54.120242 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 190/388] 10.222.0.52 () {56 vars in 1180 bytes} [Sat Sep 24 14:12:53 2022] GET /identity/v3/groups/9a8e9df9724d4fd88cc9fe6515aca1f4 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:54.125941 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8778e10-44bb-4d90-840e-44de173daf58 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:54.178272 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 199/389] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:12:54.179175 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91087df1-bb75-4635-af70-a198154365e1 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:54.225257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:54.257629 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8778e10-44bb-4d90-840e-44de173daf58 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:54.303059 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8686be6e-57dd-4493-945d-3b94141089e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:54.304027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686be6e-57dd-4493-945d-3b94141089e7 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:54.304961 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686be6e-57dd-4493-945d-3b94141089e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:54.305448 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686be6e-57dd-4493-945d-3b94141089e7 None admin] PATH_INFO: `/v3/projects/a7eea203d96a471cb590c418bb060d23` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:54.308081 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8686be6e-57dd-4493-945d-3b94141089e7 None admin] RBAC: Authorizing `identity:get_project(project_id=a7eea203d96a471cb590c418bb060d23)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:54.309748 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8686be6e-57dd-4493-945d-3b94141089e7 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:54.312450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 191/390] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:12:54 2022] GET /identity/v3/projects/a7eea203d96a471cb590c418bb060d23 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:54.318535 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8686be6e-57dd-4493-945d-3b94141089e7 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:54.335646 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91087df1-bb75-4635-af70-a198154365e1 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:54.355673 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:54.377503 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d8e24-8c92-4199-8261-da10fad5e4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:54.378201 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d8e24-8c92-4199-8261-da10fad5e4b2 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:54.378487 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d8e24-8c92-4199-8261-da10fad5e4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:54.378788 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d8e24-8c92-4199-8261-da10fad5e4b2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:54.391781 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d8e24-8c92-4199-8261-da10fad5e4b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:54.396996 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-f78d8e24-8c92-4199-8261-da10fad5e4b2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:12:54.396996 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:54.396996 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:54.396996 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:54.396996 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:54.396996 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:54.396996 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:54.396996 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:54.396996 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:54.396996 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:54.396996 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:54.396996 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:54.396996 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:54.396996 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:54.396996 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:54.396996 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:54.397856 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:54.397856 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:54.397856 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:54.397856 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:54.397856 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:54.397856 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:54.397856 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:54.397856 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:54.397856 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:54.397856 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:54.397856 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:54.397856 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:54.397856 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:54.397856 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:54.397856 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:54.397856 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:54.398631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:54.398631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:54.398631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:54.398631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:54.398631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:54.398631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:54.398631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:54.398631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:54.398631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:54.398631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:54.398631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:12:54.398631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:12:54.398631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 200/391] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:12:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:12:54.407160 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d8e24-8c92-4199-8261-da10fad5e4b2 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:54.453740 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8686be6e-57dd-4493-945d-3b94141089e7 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:54.505388 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ede90e-26c8-40aa-858f-5f30eef9413d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:54.506136 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ede90e-26c8-40aa-858f-5f30eef9413d None admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:54.506334 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ede90e-26c8-40aa-858f-5f30eef9413d None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:54.506605 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ede90e-26c8-40aa-858f-5f30eef9413d None admin] PATH_INFO: `/v3/projects/a7eea203d96a471cb590c418bb060d23/groups/9a8e9df9724d4fd88cc9fe6515aca1f4/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:54.509596 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ede90e-26c8-40aa-858f-5f30eef9413d None admin] RBAC: Authorizing `identity:create_grant(project_id=a7eea203d96a471cb590c418bb060d23, group_id=9a8e9df9724d4fd88cc9fe6515aca1f4, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:54.513214 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ede90e-26c8-40aa-858f-5f30eef9413d None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:54.545208 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 192/392] 10.222.0.52 () {58 vars in 1442 bytes} [Sat Sep 24 14:12:54 2022] PUT /identity/v3/projects/a7eea203d96a471cb590c418bb060d23/groups/9a8e9df9724d4fd88cc9fe6515aca1f4/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:12:54.568929 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d8e24-8c92-4199-8261-da10fad5e4b2 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:54.608373 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c604c205-8b33-432d-88fc-787405d3f564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:54.609046 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c604c205-8b33-432d-88fc-787405d3f564 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:54.609320 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c604c205-8b33-432d-88fc-787405d3f564 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:54.609581 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c604c205-8b33-432d-88fc-787405d3f564 None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:54.610344 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c604c205-8b33-432d-88fc-787405d3f564 None admin] RBAC: Adding query filter params (name=member) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:54.610642 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c604c205-8b33-432d-88fc-787405d3f564 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:54.612166 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c604c205-8b33-432d-88fc-787405d3f564 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:54.626830 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 201/393] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 14:12:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:54.633080 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ede90e-26c8-40aa-858f-5f30eef9413d None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:54.636578 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c604c205-8b33-432d-88fc-787405d3f564 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:54.639514 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:54.710254 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:54.838679 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ede90e-26c8-40aa-858f-5f30eef9413d None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:54.848654 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c604c205-8b33-432d-88fc-787405d3f564 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:54.887736 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19b34fe-d5c5-4940-bbcc-18680b110f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:54.888391 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b34fe-d5c5-4940-bbcc-18680b110f58 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:54.888656 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b34fe-d5c5-4940-bbcc-18680b110f58 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:54.888925 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b34fe-d5c5-4940-bbcc-18680b110f58 None admin] PATH_INFO: `/v3/groups/9a8e9df9724d4fd88cc9fe6515aca1f4` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:54.890536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19b34fe-d5c5-4940-bbcc-18680b110f58 None admin] RBAC: Authorizing `identity:get_group(group_id=9a8e9df9724d4fd88cc9fe6515aca1f4)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:54.892013 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19b34fe-d5c5-4940-bbcc-18680b110f58 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:54.893760 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 193/394] 10.222.0.52 () {56 vars in 1180 bytes} [Sat Sep 24 14:12:54 2022] GET /identity/v3/groups/9a8e9df9724d4fd88cc9fe6515aca1f4 => generated 226 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:54.901704 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19b34fe-d5c5-4940-bbcc-18680b110f58 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:54.915703 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1005ff9-615d-44b3-9506-1bcee103d25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:54.916878 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1005ff9-615d-44b3-9506-1bcee103d25d None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:54.917114 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1005ff9-615d-44b3-9506-1bcee103d25d None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:54.917400 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1005ff9-615d-44b3-9506-1bcee103d25d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:54.940265 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1005ff9-615d-44b3-9506-1bcee103d25d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:54.945522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-c1005ff9-615d-44b3-9506-1bcee103d25d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:12:54.945522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:54.945522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:54.945522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:54.945522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:54.945522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:54.945522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:54.945522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:54.945522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:54.945522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:54.945522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:54.945522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:54.945522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:54.945522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:54.945522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:54.945522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:54.946412 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:54.946412 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:54.946412 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:54.946412 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:54.946412 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:54.946412 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:54.946412 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:54.946412 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:54.946412 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:54.946412 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:54.946412 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:54.946412 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:54.946412 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:54.946412 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:54.946412 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:54.946412 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:54.947787 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:54.947787 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:54.947787 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:54.947787 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:54.947787 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:54.947787 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:54.947787 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:54.947787 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:54.947787 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:54.947787 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:54.947787 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:12:54.947787 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:12:54.951268 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 202/395] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 14:12:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:12:54.956892 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1005ff9-615d-44b3-9506-1bcee103d25d None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:54.991549 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:55.030772 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19b34fe-d5c5-4940-bbcc-18680b110f58 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:55.071217 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b55d6-0aaf-408b-98ba-e81ca1ba2adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:55.072283 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b55d6-0aaf-408b-98ba-e81ca1ba2adb None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:55.073004 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b55d6-0aaf-408b-98ba-e81ca1ba2adb None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:55.073491 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b55d6-0aaf-408b-98ba-e81ca1ba2adb None admin] PATH_INFO: `/v3/projects/c73b2a612ca646969294ac368768fcb5` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:55.074836 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083b55d6-0aaf-408b-98ba-e81ca1ba2adb None admin] RBAC: Authorizing `identity:get_project(project_id=c73b2a612ca646969294ac368768fcb5)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:55.076918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083b55d6-0aaf-408b-98ba-e81ca1ba2adb None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:55.079430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 194/396] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:12:54 2022] GET /identity/v3/projects/c73b2a612ca646969294ac368768fcb5 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:55.086032 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b55d6-0aaf-408b-98ba-e81ca1ba2adb None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:55.142032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:55.146740 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1005ff9-615d-44b3-9506-1bcee103d25d None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:55.186217 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15656b1-b88a-4504-bd2f-0ec85fcaee74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:55.187200 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15656b1-b88a-4504-bd2f-0ec85fcaee74 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:55.187655 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15656b1-b88a-4504-bd2f-0ec85fcaee74 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:55.188203 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15656b1-b88a-4504-bd2f-0ec85fcaee74 None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:55.189514 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15656b1-b88a-4504-bd2f-0ec85fcaee74 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:55.190140 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15656b1-b88a-4504-bd2f-0ec85fcaee74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:55.192155 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15656b1-b88a-4504-bd2f-0ec85fcaee74 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:55.202228 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 203/397] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:12:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:55.207569 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15656b1-b88a-4504-bd2f-0ec85fcaee74 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:55.327190 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15656b1-b88a-4504-bd2f-0ec85fcaee74 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:55.365110 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e16c01f-0f7f-4159-9331-83f9687c1476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:55.366644 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16c01f-0f7f-4159-9331-83f9687c1476 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:55.367111 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16c01f-0f7f-4159-9331-83f9687c1476 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:55.367549 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16c01f-0f7f-4159-9331-83f9687c1476 None admin] PATH_INFO: `/v3/groups/9a8e9df9724d4fd88cc9fe6515aca1f4` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:55.369567 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e16c01f-0f7f-4159-9331-83f9687c1476 None admin] RBAC: Authorizing `identity:get_group(group_id=9a8e9df9724d4fd88cc9fe6515aca1f4)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:55.371378 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e16c01f-0f7f-4159-9331-83f9687c1476 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:55.373677 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 204/398] 10.222.0.52 () {56 vars in 1180 bytes} [Sat Sep 24 14:12:55 2022] GET /identity/v3/groups/9a8e9df9724d4fd88cc9fe6515aca1f4 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:55.378945 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e16c01f-0f7f-4159-9331-83f9687c1476 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:55.418376 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b55d6-0aaf-408b-98ba-e81ca1ba2adb None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:55.451209 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:55.505922 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e16c01f-0f7f-4159-9331-83f9687c1476 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:55.523439 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37029df9-5d9a-4be0-bf6b-0ed0284b609c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:55.524544 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37029df9-5d9a-4be0-bf6b-0ed0284b609c None admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:55.524544 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37029df9-5d9a-4be0-bf6b-0ed0284b609c None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:55.524772 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37029df9-5d9a-4be0-bf6b-0ed0284b609c None admin] PATH_INFO: `/v3/projects/c73b2a612ca646969294ac368768fcb5/groups/9a8e9df9724d4fd88cc9fe6515aca1f4/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:55.532514 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37029df9-5d9a-4be0-bf6b-0ed0284b609c None admin] RBAC: Authorizing `identity:create_grant(project_id=c73b2a612ca646969294ac368768fcb5, group_id=9a8e9df9724d4fd88cc9fe6515aca1f4, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:55.533991 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37029df9-5d9a-4be0-bf6b-0ed0284b609c None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:55.550180 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af77a3-056c-4455-9e4c-0c917027b24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:55.551210 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af77a3-056c-4455-9e4c-0c917027b24a None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:55.551632 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af77a3-056c-4455-9e4c-0c917027b24a None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:55.552043 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af77a3-056c-4455-9e4c-0c917027b24a None admin] PATH_INFO: `/v3/projects/a7eea203d96a471cb590c418bb060d23` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:55.553523 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af77a3-056c-4455-9e4c-0c917027b24a None admin] RBAC: Authorizing `identity:get_project(project_id=a7eea203d96a471cb590c418bb060d23)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:55.555470 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af77a3-056c-4455-9e4c-0c917027b24a None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:55.558068 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 205/399] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:12:55 2022] GET /identity/v3/projects/a7eea203d96a471cb590c418bb060d23 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:55.559688 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:55.565664 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af77a3-056c-4455-9e4c-0c917027b24a None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:55.573874 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 195/400] 10.222.0.52 () {58 vars in 1442 bytes} [Sat Sep 24 14:12:55 2022] PUT /identity/v3/projects/c73b2a612ca646969294ac368768fcb5/groups/9a8e9df9724d4fd88cc9fe6515aca1f4/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:12:55.688309 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:55.692636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af77a3-056c-4455-9e4c-0c917027b24a None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:55.733183 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5edc89d-809f-40d3-8d08-45dd79317e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:55.734155 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edc89d-809f-40d3-8d08-45dd79317e97 None admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:55.734578 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edc89d-809f-40d3-8d08-45dd79317e97 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:55.735025 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edc89d-809f-40d3-8d08-45dd79317e97 None admin] PATH_INFO: `/v3/projects/a7eea203d96a471cb590c418bb060d23/groups/9a8e9df9724d4fd88cc9fe6515aca1f4/roles/25257d00ad2640c2a57d257b0279155f` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:55.744634 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5edc89d-809f-40d3-8d08-45dd79317e97 None admin] RBAC: Authorizing `identity:create_grant(project_id=a7eea203d96a471cb590c418bb060d23, group_id=9a8e9df9724d4fd88cc9fe6515aca1f4, role_id=25257d00ad2640c2a57d257b0279155f)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:55.746381 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5edc89d-809f-40d3-8d08-45dd79317e97 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:55.774429 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 206/401] 10.222.0.52 () {58 vars in 1442 bytes} [Sat Sep 24 14:12:55 2022] PUT /identity/v3/projects/a7eea203d96a471cb590c418bb060d23/groups/9a8e9df9724d4fd88cc9fe6515aca1f4/roles/25257d00ad2640c2a57d257b0279155f => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:12:55.785732 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c58965-0796-4acf-b04e-15b1dee256aa None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:55.786010 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c58965-0796-4acf-b04e-15b1dee256aa None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:55.786877 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c58965-0796-4acf-b04e-15b1dee256aa None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:55.787739 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 196/402] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:55.794614 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759365ef-5ac9-4642-a8d9-98748a3e25f9 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:55.795224 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759365ef-5ac9-4642-a8d9-98748a3e25f9 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:55.795891 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759365ef-5ac9-4642-a8d9-98748a3e25f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:55.821371 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-759365ef-5ac9-4642-a8d9-98748a3e25f9 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:55.823518 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-759365ef-5ac9-4642-a8d9-98748a3e25f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:55.905611 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 207/403] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:56.083331 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f2c6ef-40d5-4168-8369-d06d3181d2b6 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:56.083519 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f2c6ef-40d5-4168-8369-d06d3181d2b6 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:56.083791 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f2c6ef-40d5-4168-8369-d06d3181d2b6 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:56.085529 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 197/404] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:56.091692 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df7aa2-29a9-4d2a-9eca-ca4282a79cbc None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:56.092536 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df7aa2-29a9-4d2a-9eca-ca4282a79cbc None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:56.092985 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df7aa2-29a9-4d2a-9eca-ca4282a79cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:56.116427 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-62df7aa2-29a9-4d2a-9eca-ca4282a79cbc None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:56.117933 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-62df7aa2-29a9-4d2a-9eca-ca4282a79cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:56.204329 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 208/405] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:56.214806 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f2c6ef-40d5-4168-8369-d06d3181d2b6 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:56.350498 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f2c6ef-40d5-4168-8369-d06d3181d2b6 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:56.385814 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f77080-0c76-4738-a717-d8ccc9a67ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:56.386441 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f77080-0c76-4738-a717-d8ccc9a67ca3 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:56.386713 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f77080-0c76-4738-a717-d8ccc9a67ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:56.387011 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f77080-0c76-4738-a717-d8ccc9a67ca3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:56.397731 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f77080-0c76-4738-a717-d8ccc9a67ca3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:56.403599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-91f77080-0c76-4738-a717-d8ccc9a67ca3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:12:56.403599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:56.403599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:56.403599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:56.403599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:56.403599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:56.403599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:56.403599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:56.403599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:56.403599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:56.403599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:56.403599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:56.403599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:56.403599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:56.403599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:56.403599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:56.404641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:56.404641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:56.404641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:56.404641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:56.404641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:56.404641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:56.404641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:56.404641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:56.404641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:56.404641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:56.404641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:56.404641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:56.404641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:56.404641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:56.404641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:56.404641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:56.405621 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:56.405621 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:56.405621 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:56.405621 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:56.405621 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:56.405621 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:56.405621 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:56.405621 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:56.405621 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:56.405621 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:56.405621 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:12:56.405621 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:12:56.405621 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 198/406] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 14:12:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:12:56.410775 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df7aa2-29a9-4d2a-9eca-ca4282a79cbc None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:56.445267 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:56.563586 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df7aa2-29a9-4d2a-9eca-ca4282a79cbc None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:56.616118 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadbe754-5ad2-49db-a932-f7f7016ac765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:56.616654 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbe754-5ad2-49db-a932-f7f7016ac765 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:56.616934 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbe754-5ad2-49db-a932-f7f7016ac765 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:56.617207 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbe754-5ad2-49db-a932-f7f7016ac765 None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:56.617878 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadbe754-5ad2-49db-a932-f7f7016ac765 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:56.618182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadbe754-5ad2-49db-a932-f7f7016ac765 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:56.620144 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadbe754-5ad2-49db-a932-f7f7016ac765 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:56.630986 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 209/407] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:12:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:56.634011 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f77080-0c76-4738-a717-d8ccc9a67ca3 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:56.636330 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadbe754-5ad2-49db-a932-f7f7016ac765 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:56.700209 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:56.749522 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f77080-0c76-4738-a717-d8ccc9a67ca3 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:56.779099 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:56.788874 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb61e3c4-4e2c-4039-b18a-de85edad616f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:56.789510 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb61e3c4-4e2c-4039-b18a-de85edad616f None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:56.789788 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb61e3c4-4e2c-4039-b18a-de85edad616f None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:56.790059 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb61e3c4-4e2c-4039-b18a-de85edad616f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:56.801486 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb61e3c4-4e2c-4039-b18a-de85edad616f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:56.806949 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-fb61e3c4-4e2c-4039-b18a-de85edad616f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:12:56.806949 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:56.806949 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:56.806949 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:56.806949 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:56.806949 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:56.806949 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:56.806949 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:56.806949 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:56.806949 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:56.806949 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:56.806949 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:56.806949 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:56.806949 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:56.806949 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:56.806949 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:56.807838 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:56.807838 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:56.807838 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:56.807838 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:56.807838 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:56.807838 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:56.807838 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:56.807838 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:56.807838 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:56.807838 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:56.807838 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:56.807838 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:56.807838 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:56.807838 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:56.807838 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:56.807838 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:56.808618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:56.808618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:56.808618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:56.808618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:56.808618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:56.808618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:56.808618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:56.808618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:56.808618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:56.808618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:56.808618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:12:56.808618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:12:56.813233 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 199/408] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 14:12:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:12:56.814453 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb61e3c4-4e2c-4039-b18a-de85edad616f None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:56.847303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadbe754-5ad2-49db-a932-f7f7016ac765 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:56.921837 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb61e3c4-4e2c-4039-b18a-de85edad616f None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:56.922663 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c127488-a812-455f-ab3e-42f97c97da36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:56.923385 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c127488-a812-455f-ab3e-42f97c97da36 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:56.923662 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c127488-a812-455f-ab3e-42f97c97da36 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:56.923931 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c127488-a812-455f-ab3e-42f97c97da36 None admin] PATH_INFO: `/v3/groups/9a8e9df9724d4fd88cc9fe6515aca1f4` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:56.927601 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c127488-a812-455f-ab3e-42f97c97da36 None admin] RBAC: Authorizing `identity:get_group(group_id=9a8e9df9724d4fd88cc9fe6515aca1f4)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:56.929119 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c127488-a812-455f-ab3e-42f97c97da36 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:56.931067 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 210/409] 10.222.0.52 () {56 vars in 1180 bytes} [Sat Sep 24 14:12:56 2022] GET /identity/v3/groups/9a8e9df9724d4fd88cc9fe6515aca1f4 => generated 226 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:56.936197 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c127488-a812-455f-ab3e-42f97c97da36 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:56.939965 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:56.959626 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d129d91-2f52-4f9a-8ff0-4a7a74015865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:56.960236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d129d91-2f52-4f9a-8ff0-4a7a74015865 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:56.960516 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d129d91-2f52-4f9a-8ff0-4a7a74015865 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:56.960787 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d129d91-2f52-4f9a-8ff0-4a7a74015865 None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:56.961447 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d129d91-2f52-4f9a-8ff0-4a7a74015865 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:56.961747 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d129d91-2f52-4f9a-8ff0-4a7a74015865 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:56.963705 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d129d91-2f52-4f9a-8ff0-4a7a74015865 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:56.977457 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 200/410] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:12:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:56.979333 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d129d91-2f52-4f9a-8ff0-4a7a74015865 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:57.081654 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c127488-a812-455f-ab3e-42f97c97da36 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:57.135700 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00114472-ed2a-4060-b83b-c46d87b4b20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:57.136527 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00114472-ed2a-4060-b83b-c46d87b4b20d None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:57.136734 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00114472-ed2a-4060-b83b-c46d87b4b20d None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:57.136913 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00114472-ed2a-4060-b83b-c46d87b4b20d None admin] PATH_INFO: `/v3/projects/c73b2a612ca646969294ac368768fcb5` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:57.138226 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00114472-ed2a-4060-b83b-c46d87b4b20d None admin] RBAC: Authorizing `identity:get_project(project_id=c73b2a612ca646969294ac368768fcb5)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:57.139840 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00114472-ed2a-4060-b83b-c46d87b4b20d None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:57.141827 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 211/411] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:12:56 2022] GET /identity/v3/projects/c73b2a612ca646969294ac368768fcb5 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:57.148734 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d129d91-2f52-4f9a-8ff0-4a7a74015865 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:57.152515 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00114472-ed2a-4060-b83b-c46d87b4b20d None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:57.186056 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45b10c-80a7-4c66-bb9f-75359f93c739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:57.187041 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45b10c-80a7-4c66-bb9f-75359f93c739 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:57.187544 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45b10c-80a7-4c66-bb9f-75359f93c739 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:57.188032 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45b10c-80a7-4c66-bb9f-75359f93c739 None admin] PATH_INFO: `/v3/groups/e5a29351527643de8cac05b4be913b03` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:57.189379 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc45b10c-80a7-4c66-bb9f-75359f93c739 None admin] RBAC: Authorizing `identity:get_group(group_id=e5a29351527643de8cac05b4be913b03)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:57.191169 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc45b10c-80a7-4c66-bb9f-75359f93c739 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:57.191558 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:57.193552 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 201/412] 10.222.0.52 () {56 vars in 1180 bytes} [Sat Sep 24 14:12:56 2022] GET /identity/v3/groups/e5a29351527643de8cac05b4be913b03 => generated 229 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:57.199981 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45b10c-80a7-4c66-bb9f-75359f93c739 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:57.262853 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:57.311230 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45b10c-80a7-4c66-bb9f-75359f93c739 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:57.354889 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e158e-6163-4868-b4f4-626bdffd645a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:57.355528 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e158e-6163-4868-b4f4-626bdffd645a None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:57.355810 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e158e-6163-4868-b4f4-626bdffd645a None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:57.356076 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e158e-6163-4868-b4f4-626bdffd645a None admin] PATH_INFO: `/v3/projects/2f8134960517435ea813cbf2d41e25d7` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:57.357310 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2e158e-6163-4868-b4f4-626bdffd645a None admin] RBAC: Authorizing `identity:get_project(project_id=2f8134960517435ea813cbf2d41e25d7)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:57.358737 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2e158e-6163-4868-b4f4-626bdffd645a None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:57.360580 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 202/413] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:12:57 2022] GET /identity/v3/projects/2f8134960517435ea813cbf2d41e25d7 => generated 344 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:57.361502 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103bb19-0a25-4f8a-b869-9563984a5d53 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:57.362024 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103bb19-0a25-4f8a-b869-9563984a5d53 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:57.362303 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103bb19-0a25-4f8a-b869-9563984a5d53 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:57.364209 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 203/414] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:57.366180 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5103bb19-0a25-4f8a-b869-9563984a5d53 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:57.386523 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00114472-ed2a-4060-b83b-c46d87b4b20d None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:57.422676 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a40a80-17c9-4bee-9d1d-e97433c0ce42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:57.423661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a40a80-17c9-4bee-9d1d-e97433c0ce42 None admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:57.424086 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a40a80-17c9-4bee-9d1d-e97433c0ce42 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:57.424501 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a40a80-17c9-4bee-9d1d-e97433c0ce42 None admin] PATH_INFO: `/v3/projects/c73b2a612ca646969294ac368768fcb5/groups/9a8e9df9724d4fd88cc9fe6515aca1f4/roles/25257d00ad2640c2a57d257b0279155f` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:57.426834 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a40a80-17c9-4bee-9d1d-e97433c0ce42 None admin] RBAC: Authorizing `identity:create_grant(project_id=c73b2a612ca646969294ac368768fcb5, group_id=9a8e9df9724d4fd88cc9fe6515aca1f4, role_id=25257d00ad2640c2a57d257b0279155f)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:57.428465 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a40a80-17c9-4bee-9d1d-e97433c0ce42 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:57.448088 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 212/415] 10.222.0.52 () {58 vars in 1442 bytes} [Sat Sep 24 14:12:57 2022] PUT /identity/v3/projects/c73b2a612ca646969294ac368768fcb5/groups/9a8e9df9724d4fd88cc9fe6515aca1f4/roles/25257d00ad2640c2a57d257b0279155f => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:12:57.449857 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48633481-18f6-4ac2-bd0a-1f5597710bb4 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:57.450708 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48633481-18f6-4ac2-bd0a-1f5597710bb4 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:57.451334 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48633481-18f6-4ac2-bd0a-1f5597710bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:57.473560 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-48633481-18f6-4ac2-bd0a-1f5597710bb4 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:57.475353 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-48633481-18f6-4ac2-bd0a-1f5597710bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:57.483071 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5103bb19-0a25-4f8a-b869-9563984a5d53 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:57.504321 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:57.526137 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42864f25-9444-48f9-8e25-6db951b649d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:57.526827 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42864f25-9444-48f9-8e25-6db951b649d6 None admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:57.527116 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42864f25-9444-48f9-8e25-6db951b649d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:57.527385 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42864f25-9444-48f9-8e25-6db951b649d6 None admin] PATH_INFO: `/v3/projects/2f8134960517435ea813cbf2d41e25d7/groups/e5a29351527643de8cac05b4be913b03/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:57.529804 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42864f25-9444-48f9-8e25-6db951b649d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f8134960517435ea813cbf2d41e25d7, group_id=e5a29351527643de8cac05b4be913b03, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:57.532253 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42864f25-9444-48f9-8e25-6db951b649d6 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:57.567207 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 213/416] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:57.570180 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 204/417] 10.222.0.52 () {58 vars in 1442 bytes} [Sat Sep 24 14:12:57 2022] PUT /identity/v3/projects/2f8134960517435ea813cbf2d41e25d7/groups/e5a29351527643de8cac05b4be913b03/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:12:57.939514 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48633481-18f6-4ac2-bd0a-1f5597710bb4 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:57.943139 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:58.070102 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48633481-18f6-4ac2-bd0a-1f5597710bb4 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:58.113113 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b81213-0550-434f-8e4e-175df2f600c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:58.113661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b81213-0550-434f-8e4e-175df2f600c0 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:58.113936 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b81213-0550-434f-8e4e-175df2f600c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:58.114202 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b81213-0550-434f-8e4e-175df2f600c0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:58.129723 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b81213-0550-434f-8e4e-175df2f600c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:58.134656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-05b81213-0550-434f-8e4e-175df2f600c0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:12:58.134656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:58.134656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:58.134656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:58.134656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:58.134656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:58.134656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:58.134656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:58.134656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:58.134656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:58.134656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:58.134656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:58.134656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:58.134656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:58.134656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:58.134656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:58.135540 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:58.135540 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:58.135540 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:58.135540 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:58.135540 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:58.135540 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:58.135540 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:58.135540 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:58.135540 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:58.135540 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:58.135540 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:58.135540 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:58.135540 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:58.135540 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:58.135540 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:58.135540 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:58.136770 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:58.136770 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:58.136770 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:58.136770 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:58.136770 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:58.136770 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:58.136770 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:58.136770 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:58.136770 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:58.136770 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:58.136770 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:12:58.136770 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:12:58.136770 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 214/418] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:12:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:12:58.141676 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42864f25-9444-48f9-8e25-6db951b649d6 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:58.286555 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42864f25-9444-48f9-8e25-6db951b649d6 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:58.333068 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32686730-46ef-4efe-8315-8ffbd5d37271 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:58.333838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32686730-46ef-4efe-8315-8ffbd5d37271 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:58.334447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32686730-46ef-4efe-8315-8ffbd5d37271 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:58.335044 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de12f8-1957-4ff8-a6e9-866f8a093073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:58.335667 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de12f8-1957-4ff8-a6e9-866f8a093073 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:58.335947 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de12f8-1957-4ff8-a6e9-866f8a093073 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:58.336220 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de12f8-1957-4ff8-a6e9-866f8a093073 None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:58.336891 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11de12f8-1957-4ff8-a6e9-866f8a093073 None admin] RBAC: Adding query filter params (name=member) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:58.337227 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11de12f8-1957-4ff8-a6e9-866f8a093073 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:58.338166 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 215/419] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:58.340448 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11de12f8-1957-4ff8-a6e9-866f8a093073 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:58.343431 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb48e7c-4568-48f3-8563-09a80d0a31ff None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:58.344042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb48e7c-4568-48f3-8563-09a80d0a31ff None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:58.344423 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb48e7c-4568-48f3-8563-09a80d0a31ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:58.354119 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 205/420] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 14:12:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:58.359866 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de12f8-1957-4ff8-a6e9-866f8a093073 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:58.365424 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-dcb48e7c-4568-48f3-8563-09a80d0a31ff None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:58.366657 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-dcb48e7c-4568-48f3-8563-09a80d0a31ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:58.433447 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:58.439273 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:58.448658 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 216/421] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:12:58.544485 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de12f8-1957-4ff8-a6e9-866f8a093073 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:58.594464 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823f0312-f5d6-426c-94c6-d32b5453a064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:58.595100 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f0312-f5d6-426c-94c6-d32b5453a064 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:58.595377 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f0312-f5d6-426c-94c6-d32b5453a064 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:58.595645 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f0312-f5d6-426c-94c6-d32b5453a064 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:58.668815 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823f0312-f5d6-426c-94c6-d32b5453a064 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:58.670446 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823f0312-f5d6-426c-94c6-d32b5453a064 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:58.711648 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:58.788940 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 206/422] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:12:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 24 14:12:58.795438 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb48e7c-4568-48f3-8563-09a80d0a31ff None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:58.822001 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb148b-64fa-495f-94ef-918db6a68589 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:58.822354 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb148b-64fa-495f-94ef-918db6a68589 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:58.822581 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb148b-64fa-495f-94ef-918db6a68589 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:58.824167 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 207/423] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:12:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:12:58.836047 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60ea61-4b88-4ab3-aec5-40b09b8ca2bf None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:58.836554 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60ea61-4b88-4ab3-aec5-40b09b8ca2bf None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:58.837015 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60ea61-4b88-4ab3-aec5-40b09b8ca2bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:58.870859 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-2b60ea61-4b88-4ab3-aec5-40b09b8ca2bf None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:12:58.872053 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-2b60ea61-4b88-4ab3-aec5-40b09b8ca2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:12:58.927638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb48e7c-4568-48f3-8563-09a80d0a31ff None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:58.965585 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2039c60-49dc-49b7-82a0-d4b02dd56923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:58.965878 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 208/424] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:12:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:12:58.966693 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2039c60-49dc-49b7-82a0-d4b02dd56923 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:58.966797 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b60ea61-4b88-4ab3-aec5-40b09b8ca2bf None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:58.967308 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2039c60-49dc-49b7-82a0-d4b02dd56923 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:58.967763 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2039c60-49dc-49b7-82a0-d4b02dd56923 None admin] PATH_INFO: `/v3/projects/a7eea203d96a471cb590c418bb060d23` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:58.971595 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2039c60-49dc-49b7-82a0-d4b02dd56923 None admin] RBAC: Authorizing `identity:get_project(project_id=a7eea203d96a471cb590c418bb060d23)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:58.975300 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2039c60-49dc-49b7-82a0-d4b02dd56923 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:58.982953 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 217/425] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:12:58 2022] GET /identity/v3/projects/a7eea203d96a471cb590c418bb060d23 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:58.984013 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2039c60-49dc-49b7-82a0-d4b02dd56923 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:59.089827 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:59.095752 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:59.154799 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b60ea61-4b88-4ab3-aec5-40b09b8ca2bf None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:59.171217 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2039c60-49dc-49b7-82a0-d4b02dd56923 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:59.214943 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b24e9-1cda-4b1f-9591-c2d55f5442a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:59.215657 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b24e9-1cda-4b1f-9591-c2d55f5442a5 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:59.215933 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b24e9-1cda-4b1f-9591-c2d55f5442a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:59.216257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b24e9-1cda-4b1f-9591-c2d55f5442a5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:59.230839 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7b24e9-1cda-4b1f-9591-c2d55f5442a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:59.231908 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-0c7b24e9-1cda-4b1f-9591-c2d55f5442a5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:12:59.231908 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:59.231908 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:59.231908 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:59.231908 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:59.231908 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:59.231908 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:59.231908 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:59.231908 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:59.231908 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:59.231908 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:59.231908 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:59.231908 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:59.231908 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:59.231908 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:59.231908 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:59.232752 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:59.232752 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:59.232752 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:59.232752 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:59.232752 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:59.232752 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:59.232752 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:59.232752 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:59.232752 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:59.232752 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:59.232752 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:59.232752 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:59.232752 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:59.232752 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:59.232752 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:59.232752 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:59.233629 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:59.233629 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:59.233629 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:59.233629 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:59.233629 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:59.233629 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:59.233629 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:59.233629 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:59.233629 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:59.233629 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:59.233629 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 14:12:59.233629 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:12:59.233629 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 209/426] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:12:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:12:59.240553 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b24e9-1cda-4b1f-9591-c2d55f5442a5 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:59.241479 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a9bb49-7d0d-4e1d-a457-dc5d914d262e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:59.242285 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9bb49-7d0d-4e1d-a457-dc5d914d262e None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:59.242710 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9bb49-7d0d-4e1d-a457-dc5d914d262e None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:59.243170 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9bb49-7d0d-4e1d-a457-dc5d914d262e None admin] PATH_INFO: `/v3/groups/9a8e9df9724d4fd88cc9fe6515aca1f4` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:59.247601 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a9bb49-7d0d-4e1d-a457-dc5d914d262e None admin] RBAC: Authorizing `identity:get_group(group_id=9a8e9df9724d4fd88cc9fe6515aca1f4)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:59.249070 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a9bb49-7d0d-4e1d-a457-dc5d914d262e None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:59.250972 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 218/427] 10.222.0.52 () {56 vars in 1180 bytes} [Sat Sep 24 14:12:58 2022] GET /identity/v3/groups/9a8e9df9724d4fd88cc9fe6515aca1f4 => generated 226 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:59.260271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a9bb49-7d0d-4e1d-a457-dc5d914d262e None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:59.344571 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b24e9-1cda-4b1f-9591-c2d55f5442a5 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:59.373091 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:59.383100 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199dfcbc-7fd9-40b4-ac44-a1f382678514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:59.383659 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199dfcbc-7fd9-40b4-ac44-a1f382678514 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:59.384003 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199dfcbc-7fd9-40b4-ac44-a1f382678514 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:59.384344 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199dfcbc-7fd9-40b4-ac44-a1f382678514 None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:59.385662 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199dfcbc-7fd9-40b4-ac44-a1f382678514 None admin] RBAC: Adding query filter params (name=member) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:59.386027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199dfcbc-7fd9-40b4-ac44-a1f382678514 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:59.387535 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199dfcbc-7fd9-40b4-ac44-a1f382678514 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:59.396913 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 210/428] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 14:12:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:59.397788 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199dfcbc-7fd9-40b4-ac44-a1f382678514 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:59.429767 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a9bb49-7d0d-4e1d-a457-dc5d914d262e None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:59.462503 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:59.469822 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb72f0c-ecad-4f2f-9c65-9376d3b555d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:59.470611 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb72f0c-ecad-4f2f-9c65-9376d3b555d0 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:59.471295 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb72f0c-ecad-4f2f-9c65-9376d3b555d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:59.472333 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb72f0c-ecad-4f2f-9c65-9376d3b555d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:59.473117 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb72f0c-ecad-4f2f-9c65-9376d3b555d0 None admin] RBAC: Adding query filter params (group.id=9a8e9df9724d4fd88cc9fe6515aca1f4, role.id=640af5182e6848498c16a19cddbfb738, scope.project.id=a7eea203d96a471cb590c418bb060d23) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:59.473851 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb72f0c-ecad-4f2f-9c65-9376d3b555d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:59.475631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb72f0c-ecad-4f2f-9c65-9376d3b555d0 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:59.483085 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 219/429] 10.222.0.52 () {56 vars in 1376 bytes} [Sat Sep 24 14:12:59 2022] GET /identity/v3/role_assignments?group.id=9a8e9df9724d4fd88cc9fe6515aca1f4&scope.project.id=a7eea203d96a471cb590c418bb060d23&role.id=640af5182e6848498c16a19cddbfb738 => generated 611 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:12:59.484147 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb72f0c-ecad-4f2f-9c65-9376d3b555d0 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:59.544017 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199dfcbc-7fd9-40b4-ac44-a1f382678514 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:59.599521 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb01e0-bf2f-4d79-90b4-24449a646dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:59.600167 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb01e0-bf2f-4d79-90b4-24449a646dbc None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:59.600443 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb01e0-bf2f-4d79-90b4-24449a646dbc None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:59.600707 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb01e0-bf2f-4d79-90b4-24449a646dbc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:59.617055 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb72f0c-ecad-4f2f-9c65-9376d3b555d0 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:59.629957 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb01e0-bf2f-4d79-90b4-24449a646dbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:12:59.636186 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-f2cb01e0-bf2f-4d79-90b4-24449a646dbc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:12:59.636186 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:12:59.636186 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:12:59.636186 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:12:59.636186 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:12:59.636186 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:12:59.636186 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:12:59.636186 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:12:59.636186 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:12:59.636186 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:12:59.636186 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:12:59.636186 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:12:59.636186 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:12:59.636186 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:12:59.636186 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:12:59.636186 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:12:59.637118 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:12:59.637118 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:12:59.637118 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:12:59.637118 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:12:59.637118 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:12:59.637118 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:12:59.637118 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:12:59.637118 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:12:59.637118 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:12:59.637118 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:12:59.637118 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:12:59.637118 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:12:59.637118 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:12:59.637118 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:12:59.637118 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:12:59.637118 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:12:59.637898 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:12:59.637898 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:12:59.637898 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:12:59.637898 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:12:59.637898 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:12:59.637898 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:12:59.637898 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:12:59.637898 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:12:59.637898 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:12:59.637898 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:12:59.637898 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:12:59.637898 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:12:59.637898 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 211/430] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 14:12:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:12:59.643817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb01e0-bf2f-4d79-90b4-24449a646dbc None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:59.704376 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb63e00c-fd2b-4d00-8fa7-4e0939fecc1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:59.705965 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63e00c-fd2b-4d00-8fa7-4e0939fecc1a None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:59.706246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63e00c-fd2b-4d00-8fa7-4e0939fecc1a None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:59.706515 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63e00c-fd2b-4d00-8fa7-4e0939fecc1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:59.736535 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:59.809638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:12:59.821380 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb63e00c-fd2b-4d00-8fa7-4e0939fecc1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:59.823721 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb63e00c-fd2b-4d00-8fa7-4e0939fecc1a None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:59.824245 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb01e0-bf2f-4d79-90b4-24449a646dbc None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:12:59.874298 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c20f9-cabe-4a93-90f4-4552d8bc9912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:12:59.875141 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c20f9-cabe-4a93-90f4-4552d8bc9912 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:12:59.875516 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c20f9-cabe-4a93-90f4-4552d8bc9912 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:12:59.875882 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c20f9-cabe-4a93-90f4-4552d8bc9912 None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:12:59.876705 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1c20f9-cabe-4a93-90f4-4552d8bc9912 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:12:59.877005 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1c20f9-cabe-4a93-90f4-4552d8bc9912 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:12:59.879012 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1c20f9-cabe-4a93-90f4-4552d8bc9912 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:12:59.890118 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 212/431] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:12:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:12:59.896433 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c20f9-cabe-4a93-90f4-4552d8bc9912 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:12:59.958983 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 220/432] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:12:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:12:59.962101 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb63e00c-fd2b-4d00-8fa7-4e0939fecc1a None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:00.019367 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c20f9-cabe-4a93-90f4-4552d8bc9912 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:00.037018 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:00.066766 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9518f8da-fba0-4fa6-9c14-36f4f034a844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:00.067501 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518f8da-fba0-4fa6-9c14-36f4f034a844 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:00.067783 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518f8da-fba0-4fa6-9c14-36f4f034a844 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:00.068153 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518f8da-fba0-4fa6-9c14-36f4f034a844 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:00.070499 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:00.079811 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb63e00c-fd2b-4d00-8fa7-4e0939fecc1a None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:00.124027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dde8c9-6370-47e5-a1b6-401c69fa7ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:00.124991 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dde8c9-6370-47e5-a1b6-401c69fa7ebf None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:00.125457 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dde8c9-6370-47e5-a1b6-401c69fa7ebf None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:00.125825 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dde8c9-6370-47e5-a1b6-401c69fa7ebf None admin] PATH_INFO: `/v3/projects/c73b2a612ca646969294ac368768fcb5` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:00.127036 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dde8c9-6370-47e5-a1b6-401c69fa7ebf None admin] RBAC: Authorizing `identity:get_project(project_id=c73b2a612ca646969294ac368768fcb5)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:00.128480 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dde8c9-6370-47e5-a1b6-401c69fa7ebf None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:00.130646 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 221/433] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:12:59 2022] GET /identity/v3/projects/c73b2a612ca646969294ac368768fcb5 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:13:00.131636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0173185-d960-4d41-a540-dde4648c2557 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:00.131905 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0173185-d960-4d41-a540-dde4648c2557 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:00.132178 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0173185-d960-4d41-a540-dde4648c2557 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:00.133609 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 222/434] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:00.138984 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0173185-d960-4d41-a540-dde4648c2557 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:00.156388 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9518f8da-fba0-4fa6-9c14-36f4f034a844 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:00.158626 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9518f8da-fba0-4fa6-9c14-36f4f034a844 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:00.252871 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0173185-d960-4d41-a540-dde4648c2557 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:00.267330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 213/435] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:12:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 24 14:13:00.272064 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d36ac-08b4-4604-b2f3-8bb1c2c1bfc9 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:00.272064 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d36ac-08b4-4604-b2f3-8bb1c2c1bfc9 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:00.272064 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d36ac-08b4-4604-b2f3-8bb1c2c1bfc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:00.286674 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-6e8d36ac-08b4-4604-b2f3-8bb1c2c1bfc9 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:00.287894 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-6e8d36ac-08b4-4604-b2f3-8bb1c2c1bfc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:00.291444 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8816ff2-ae01-4f61-a74e-ebc512478256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:00.292065 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8816ff2-ae01-4f61-a74e-ebc512478256 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:00.292337 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8816ff2-ae01-4f61-a74e-ebc512478256 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:00.292605 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8816ff2-ae01-4f61-a74e-ebc512478256 None admin] PATH_INFO: `/v3/groups/9a8e9df9724d4fd88cc9fe6515aca1f4` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:00.293693 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8816ff2-ae01-4f61-a74e-ebc512478256 None admin] RBAC: Authorizing `identity:get_group(group_id=9a8e9df9724d4fd88cc9fe6515aca1f4)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:00.295145 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8816ff2-ae01-4f61-a74e-ebc512478256 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:00.295741 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:00.296855 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 223/436] 10.222.0.52 () {56 vars in 1180 bytes} [Sat Sep 24 14:13:00 2022] GET /identity/v3/groups/9a8e9df9724d4fd88cc9fe6515aca1f4 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:00.298086 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53306e55-737b-4ebb-a140-e78498d1b8ec None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:00.298364 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53306e55-737b-4ebb-a140-e78498d1b8ec None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:00.298657 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53306e55-737b-4ebb-a140-e78498d1b8ec None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:00.299877 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 224/437] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:00.300563 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53306e55-737b-4ebb-a140-e78498d1b8ec None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:00.331003 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:00.359750 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 214/438] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:13:00.360697 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8d36ac-08b4-4604-b2f3-8bb1c2c1bfc9 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:00.406247 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53306e55-737b-4ebb-a140-e78498d1b8ec None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:00.442170 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caece2df-65fc-41d0-b1b6-b199a48078d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:00.442796 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caece2df-65fc-41d0-b1b6-b199a48078d8 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:00.443114 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caece2df-65fc-41d0-b1b6-b199a48078d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:00.443390 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caece2df-65fc-41d0-b1b6-b199a48078d8 None admin] PATH_INFO: `/v3/projects/a7eea203d96a471cb590c418bb060d23` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:00.444507 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caece2df-65fc-41d0-b1b6-b199a48078d8 None admin] RBAC: Authorizing `identity:get_project(project_id=a7eea203d96a471cb590c418bb060d23)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:00.445960 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caece2df-65fc-41d0-b1b6-b199a48078d8 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:00.448121 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 225/439] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:13:00 2022] GET /identity/v3/projects/a7eea203d96a471cb590c418bb060d23 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:00.449199 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987ea4c-aa80-48b0-ab69-f5213019a1b7 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:00.449477 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987ea4c-aa80-48b0-ab69-f5213019a1b7 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:00.449746 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987ea4c-aa80-48b0-ab69-f5213019a1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:00.468407 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-e987ea4c-aa80-48b0-ab69-f5213019a1b7 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:00.468659 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8d36ac-08b4-4604-b2f3-8bb1c2c1bfc9 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:00.469579 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-e987ea4c-aa80-48b0-ab69-f5213019a1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:00.505404 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b18734-2118-4cb9-824d-ed35e6549b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:00.506164 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b18734-2118-4cb9-824d-ed35e6549b6f None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:00.506469 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b18734-2118-4cb9-824d-ed35e6549b6f None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:00.506721 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b18734-2118-4cb9-824d-ed35e6549b6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:00.507485 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b18734-2118-4cb9-824d-ed35e6549b6f None admin] RBAC: Adding query filter params (group.id=9a8e9df9724d4fd88cc9fe6515aca1f4, role.id=640af5182e6848498c16a19cddbfb738, scope.project.id=c73b2a612ca646969294ac368768fcb5) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:00.507793 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b18734-2118-4cb9-824d-ed35e6549b6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:00.509224 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b18734-2118-4cb9-824d-ed35e6549b6f None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:00.522930 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 215/440] 10.222.0.52 () {56 vars in 1376 bytes} [Sat Sep 24 14:13:00 2022] GET /identity/v3/role_assignments?group.id=9a8e9df9724d4fd88cc9fe6515aca1f4&scope.project.id=c73b2a612ca646969294ac368768fcb5&role.id=640af5182e6848498c16a19cddbfb738 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:00.522930 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b18734-2118-4cb9-824d-ed35e6549b6f None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:00.533935 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:00.567374 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 226/441] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:13:00.636441 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e987ea4c-aa80-48b0-ab69-f5213019a1b7 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:00.638248 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b18734-2118-4cb9-824d-ed35e6549b6f None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:00.676490 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:00.689770 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a9b2ad-530a-4ce8-8437-ef2f2ec12435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:00.690417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a9b2ad-530a-4ce8-8437-ef2f2ec12435 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:00.690695 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a9b2ad-530a-4ce8-8437-ef2f2ec12435 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:00.691076 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a9b2ad-530a-4ce8-8437-ef2f2ec12435 None admin] PATH_INFO: `/v3/groups/9a8e9df9724d4fd88cc9fe6515aca1f4` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:00.692293 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a9b2ad-530a-4ce8-8437-ef2f2ec12435 None admin] RBAC: Authorizing `identity:get_group(group_id=9a8e9df9724d4fd88cc9fe6515aca1f4)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:00.693730 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a9b2ad-530a-4ce8-8437-ef2f2ec12435 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:00.696276 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 216/442] 10.222.0.52 () {56 vars in 1180 bytes} [Sat Sep 24 14:13:00 2022] GET /identity/v3/groups/9a8e9df9724d4fd88cc9fe6515aca1f4 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:00.702552 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a9b2ad-530a-4ce8-8437-ef2f2ec12435 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:00.779799 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e987ea4c-aa80-48b0-ab69-f5213019a1b7 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:00.796480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:00.822958 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0242be26-9783-4541-8cab-609556dbf9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:00.823892 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242be26-9783-4541-8cab-609556dbf9f0 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:00.824319 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242be26-9783-4541-8cab-609556dbf9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:00.824777 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242be26-9783-4541-8cab-609556dbf9f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:00.833139 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a9b2ad-530a-4ce8-8437-ef2f2ec12435 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:00.836876 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0242be26-9783-4541-8cab-609556dbf9f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:13:00.841796 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-0242be26-9783-4541-8cab-609556dbf9f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:13:00.841796 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:00.841796 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:00.841796 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:00.841796 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:00.841796 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:00.841796 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:00.841796 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:00.841796 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:00.841796 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:00.841796 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:00.841796 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:00.841796 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:13:00.841796 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:13:00.841796 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:13:00.841796 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:13:00.842832 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:13:00.842832 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:13:00.842832 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:13:00.842832 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:13:00.842832 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:00.842832 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:00.842832 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:00.842832 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:00.842832 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:00.842832 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:00.842832 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:00.842832 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:00.842832 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:00.842832 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:00.842832 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:00.842832 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:00.843963 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:00.843963 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:00.843963 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:00.843963 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:00.843963 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:13:00.843963 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:13:00.843963 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:13:00.843963 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:13:00.843963 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:13:00.843963 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:13:00.843963 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:13:00.843963 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:00.843963 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 227/443] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 14:13:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:00.843963 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0242be26-9783-4541-8cab-609556dbf9f0 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:00.867227 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f5ae7-e496-447b-99be-81ec1b8c7c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:00.867833 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f5ae7-e496-447b-99be-81ec1b8c7c74 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:00.868112 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f5ae7-e496-447b-99be-81ec1b8c7c74 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:00.868381 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f5ae7-e496-447b-99be-81ec1b8c7c74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:00.869628 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f5ae7-e496-447b-99be-81ec1b8c7c74 None admin] RBAC: Adding query filter params (group.id=9a8e9df9724d4fd88cc9fe6515aca1f4, role.id=25257d00ad2640c2a57d257b0279155f, scope.project.id=a7eea203d96a471cb590c418bb060d23) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:00.869931 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f5ae7-e496-447b-99be-81ec1b8c7c74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:00.871421 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f5ae7-e496-447b-99be-81ec1b8c7c74 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:00.877198 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 217/444] 10.222.0.52 () {56 vars in 1376 bytes} [Sat Sep 24 14:13:00 2022] GET /identity/v3/role_assignments?group.id=9a8e9df9724d4fd88cc9fe6515aca1f4&scope.project.id=a7eea203d96a471cb590c418bb060d23&role.id=25257d00ad2640c2a57d257b0279155f => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:13:00.877958 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f5ae7-e496-447b-99be-81ec1b8c7c74 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:00.947406 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:00.962023 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0242be26-9783-4541-8cab-609556dbf9f0 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:00.995145 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f5ae7-e496-447b-99be-81ec1b8c7c74 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:01.001177 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0162a5a-0260-4b1c-a6a7-761b3c4e3518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:01.001817 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0162a5a-0260-4b1c-a6a7-761b3c4e3518 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:01.002090 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0162a5a-0260-4b1c-a6a7-761b3c4e3518 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:01.002356 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0162a5a-0260-4b1c-a6a7-761b3c4e3518 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:01.014103 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0162a5a-0260-4b1c-a6a7-761b3c4e3518 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:13:01.019188 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-b0162a5a-0260-4b1c-a6a7-761b3c4e3518 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:13:01.019188 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:01.019188 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:01.019188 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:01.019188 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:01.019188 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:01.019188 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:01.019188 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:01.019188 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:01.019188 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:01.019188 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:01.019188 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:01.019188 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:13:01.019188 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:13:01.019188 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:13:01.019188 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:13:01.020132 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:13:01.020132 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:13:01.020132 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:13:01.020132 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:13:01.020132 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:01.020132 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:01.020132 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:01.020132 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:01.020132 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:01.020132 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:01.020132 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:01.020132 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:01.020132 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:01.020132 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:01.020132 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:01.020132 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:01.020908 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:01.020908 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:01.020908 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:01.020908 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:01.020908 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:13:01.020908 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:13:01.020908 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:13:01.020908 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:13:01.020908 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:13:01.020908 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:13:01.020908 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:13:01.020908 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:01.020908 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 228/445] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 14:13:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:01.026399 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0162a5a-0260-4b1c-a6a7-761b3c4e3518 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:01.036614 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f0994-a4bb-4633-9f28-7be1f8897457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:01.037160 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f0994-a4bb-4633-9f28-7be1f8897457 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:01.037435 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f0994-a4bb-4633-9f28-7be1f8897457 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:01.037708 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f0994-a4bb-4633-9f28-7be1f8897457 None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:01.038383 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936f0994-a4bb-4633-9f28-7be1f8897457 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:01.038708 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936f0994-a4bb-4633-9f28-7be1f8897457 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:01.040109 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936f0994-a4bb-4633-9f28-7be1f8897457 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:01.046409 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:01.053076 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 218/446] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:13:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:01.059130 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f0994-a4bb-4633-9f28-7be1f8897457 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:01.145171 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0162a5a-0260-4b1c-a6a7-761b3c4e3518 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:01.173608 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f0994-a4bb-4633-9f28-7be1f8897457 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:01.186064 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede26fb-6b5c-4a4a-936b-4c99ddae0b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:01.186676 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede26fb-6b5c-4a4a-936b-4c99ddae0b7a None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:01.186994 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede26fb-6b5c-4a4a-936b-4c99ddae0b7a None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:01.187261 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede26fb-6b5c-4a4a-936b-4c99ddae0b7a None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:01.187931 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eede26fb-6b5c-4a4a-936b-4c99ddae0b7a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:01.188227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eede26fb-6b5c-4a4a-936b-4c99ddae0b7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:01.189601 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eede26fb-6b5c-4a4a-936b-4c99ddae0b7a None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:01.204652 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 229/447] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:13:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:13:01.206027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede26fb-6b5c-4a4a-936b-4c99ddae0b7a None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:01.214752 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf2c7a-e450-40d8-a883-3ed274b4cfc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:01.215532 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf2c7a-e450-40d8-a883-3ed274b4cfc7 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:01.215784 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf2c7a-e450-40d8-a883-3ed274b4cfc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:01.216051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf2c7a-e450-40d8-a883-3ed274b4cfc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:01.221532 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:01.291999 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:01.295395 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaf2c7a-e450-40d8-a883-3ed274b4cfc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:01.297062 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaf2c7a-e450-40d8-a883-3ed274b4cfc7 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:01.324039 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede26fb-6b5c-4a4a-936b-4c99ddae0b7a None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:01.362861 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da6a6ff-a86e-4e88-a664-24853c199b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:01.363709 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da6a6ff-a86e-4e88-a664-24853c199b80 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:01.364166 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da6a6ff-a86e-4e88-a664-24853c199b80 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:01.364440 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da6a6ff-a86e-4e88-a664-24853c199b80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:01.411370 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 219/448] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:13:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 24 14:13:01.418563 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf2c7a-e450-40d8-a883-3ed274b4cfc7 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:01.440612 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da6a6ff-a86e-4e88-a664-24853c199b80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:01.442074 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da6a6ff-a86e-4e88-a664-24853c199b80 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:01.453646 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:01.515097 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:01.518169 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf2c7a-e450-40d8-a883-3ed274b4cfc7 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:01.539812 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 230/449] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:13:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 24 14:13:01.546458 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da6a6ff-a86e-4e88-a664-24853c199b80 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:01.552890 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d69ffe-3585-47ff-bca9-4eb8f58371ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:01.553517 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d69ffe-3585-47ff-bca9-4eb8f58371ab None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:01.553795 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d69ffe-3585-47ff-bca9-4eb8f58371ab None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:01.554063 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d69ffe-3585-47ff-bca9-4eb8f58371ab None admin] PATH_INFO: `/v3/projects/2f8134960517435ea813cbf2d41e25d7` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:01.555211 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d69ffe-3585-47ff-bca9-4eb8f58371ab None admin] RBAC: Authorizing `identity:get_project(project_id=2f8134960517435ea813cbf2d41e25d7)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:01.556641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d69ffe-3585-47ff-bca9-4eb8f58371ab None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:01.558389 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 220/450] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:13:01 2022] GET /identity/v3/projects/2f8134960517435ea813cbf2d41e25d7 => generated 344 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:13:01.563750 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d69ffe-3585-47ff-bca9-4eb8f58371ab None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:01.640803 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da6a6ff-a86e-4e88-a664-24853c199b80 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:01.651448 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:01.659104 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d69ffe-3585-47ff-bca9-4eb8f58371ab None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:01.676120 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f24e78b-5ee1-43a3-aca1-ffed98a6148c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:01.676745 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f24e78b-5ee1-43a3-aca1-ffed98a6148c None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:01.677018 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f24e78b-5ee1-43a3-aca1-ffed98a6148c None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:01.677286 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f24e78b-5ee1-43a3-aca1-ffed98a6148c None admin] PATH_INFO: `/v3/projects/c73b2a612ca646969294ac368768fcb5` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:01.678388 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f24e78b-5ee1-43a3-aca1-ffed98a6148c None admin] RBAC: Authorizing `identity:get_project(project_id=c73b2a612ca646969294ac368768fcb5)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:01.679922 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f24e78b-5ee1-43a3-aca1-ffed98a6148c None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:01.681529 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 231/451] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:13:01 2022] GET /identity/v3/projects/c73b2a612ca646969294ac368768fcb5 => generated 302 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:13:01.686963 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f24e78b-5ee1-43a3-aca1-ffed98a6148c None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:01.691901 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf2bc24-0782-4b3e-a34c-7f1afbaa3245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:01.692521 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2bc24-0782-4b3e-a34c-7f1afbaa3245 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:01.692796 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2bc24-0782-4b3e-a34c-7f1afbaa3245 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:01.693061 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2bc24-0782-4b3e-a34c-7f1afbaa3245 None admin] PATH_INFO: `/v3/groups/e5a29351527643de8cac05b4be913b03` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:01.694153 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf2bc24-0782-4b3e-a34c-7f1afbaa3245 None admin] RBAC: Authorizing `identity:get_group(group_id=e5a29351527643de8cac05b4be913b03)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:01.695604 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf2bc24-0782-4b3e-a34c-7f1afbaa3245 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:01.702425 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 221/452] 10.222.0.52 () {56 vars in 1180 bytes} [Sat Sep 24 14:13:01 2022] GET /identity/v3/groups/e5a29351527643de8cac05b4be913b03 => generated 229 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:13:01.703235 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf2bc24-0782-4b3e-a34c-7f1afbaa3245 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:01.736061 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:01.787519 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f24e78b-5ee1-43a3-aca1-ffed98a6148c None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:01.798981 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf2bc24-0782-4b3e-a34c-7f1afbaa3245 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:01.819814 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ef001b-735b-421c-8b18-63c58ef348f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:01.820439 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef001b-735b-421c-8b18-63c58ef348f2 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:01.820711 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef001b-735b-421c-8b18-63c58ef348f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:01.820982 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef001b-735b-421c-8b18-63c58ef348f2 None admin] PATH_INFO: `/v3/groups/9a8e9df9724d4fd88cc9fe6515aca1f4` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:01.822591 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ef001b-735b-421c-8b18-63c58ef348f2 None admin] RBAC: Authorizing `identity:get_group(group_id=9a8e9df9724d4fd88cc9fe6515aca1f4)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:01.824059 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ef001b-735b-421c-8b18-63c58ef348f2 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:01.825596 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 232/453] 10.222.0.52 () {56 vars in 1180 bytes} [Sat Sep 24 14:13:01 2022] GET /identity/v3/groups/9a8e9df9724d4fd88cc9fe6515aca1f4 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:13:01.831689 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ef001b-735b-421c-8b18-63c58ef348f2 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:01.836614 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce51058-4632-4689-a0f5-17d6556195bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:01.837221 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce51058-4632-4689-a0f5-17d6556195bb None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:01.837501 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce51058-4632-4689-a0f5-17d6556195bb None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:01.838453 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce51058-4632-4689-a0f5-17d6556195bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:01.839243 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce51058-4632-4689-a0f5-17d6556195bb None admin] RBAC: Adding query filter params (group.id=e5a29351527643de8cac05b4be913b03, role.id=d00b78c32dbf4cc7925a55bb621d8fcf, scope.project.id=2f8134960517435ea813cbf2d41e25d7) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:01.839553 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce51058-4632-4689-a0f5-17d6556195bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:01.840995 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce51058-4632-4689-a0f5-17d6556195bb None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:01.846782 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 222/454] 10.222.0.52 () {56 vars in 1376 bytes} [Sat Sep 24 14:13:01 2022] GET /identity/v3/role_assignments?group.id=e5a29351527643de8cac05b4be913b03&scope.project.id=2f8134960517435ea813cbf2d41e25d7&role.id=d00b78c32dbf4cc7925a55bb621d8fcf => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:13:01.881931 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:01.938281 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ef001b-735b-421c-8b18-63c58ef348f2 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:01.972857 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb4056-f932-4ef2-b477-3c9d009b7717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:01.973472 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb4056-f932-4ef2-b477-3c9d009b7717 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:01.973755 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb4056-f932-4ef2-b477-3c9d009b7717 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:01.974020 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb4056-f932-4ef2-b477-3c9d009b7717 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:01.974752 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afb4056-f932-4ef2-b477-3c9d009b7717 None admin] RBAC: Adding query filter params (group.id=9a8e9df9724d4fd88cc9fe6515aca1f4, role.id=25257d00ad2640c2a57d257b0279155f, scope.project.id=c73b2a612ca646969294ac368768fcb5) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:01.975091 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afb4056-f932-4ef2-b477-3c9d009b7717 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:01.976543 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afb4056-f932-4ef2-b477-3c9d009b7717 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:01.983457 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 233/455] 10.222.0.52 () {56 vars in 1376 bytes} [Sat Sep 24 14:13:01 2022] GET /identity/v3/role_assignments?group.id=9a8e9df9724d4fd88cc9fe6515aca1f4&scope.project.id=c73b2a612ca646969294ac368768fcb5&role.id=25257d00ad2640c2a57d257b0279155f => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:04.918429 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd4bc5-49bd-412b-9f93-fd9f5c064f89 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:04.919467 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd4bc5-49bd-412b-9f93-fd9f5c064f89 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:04.919943 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd4bc5-49bd-412b-9f93-fd9f5c064f89 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:04.921787 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 223/456] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:04.926970 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a162f43-0448-4785-aa9f-a7ba3246809f None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:04.927390 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a162f43-0448-4785-aa9f-a7ba3246809f None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:04.927576 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a162f43-0448-4785-aa9f-a7ba3246809f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:04.950256 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d7e02-16f1-4f21-a01d-e645f032a374 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:04.950914 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d7e02-16f1-4f21-a01d-e645f032a374 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:04.951393 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d7e02-16f1-4f21-a01d-e645f032a374 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:04.952816 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-5a162f43-0448-4785-aa9f-a7ba3246809f None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:04.953593 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 224/457] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:04.955040 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-5a162f43-0448-4785-aa9f-a7ba3246809f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:04.958266 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6185e1f-fb46-4c54-a73f-6268ce5baf56 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:04.958541 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6185e1f-fb46-4c54-a73f-6268ce5baf56 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:04.958807 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6185e1f-fb46-4c54-a73f-6268ce5baf56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:04.987977 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-f6185e1f-fb46-4c54-a73f-6268ce5baf56 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:04.989287 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-f6185e1f-fb46-4c54-a73f-6268ce5baf56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:05.035282 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 234/458] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:13:05.085491 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 225/459] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:13:05.314793 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf713e8-fe1d-4931-953a-181c7ce0883c None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:05.315183 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf713e8-fe1d-4931-953a-181c7ce0883c None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:05.315403 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf713e8-fe1d-4931-953a-181c7ce0883c None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:05.316979 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 235/460] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:05.321834 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d621d10-c1a9-45a7-82f3-612aaf04a51c None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:05.322157 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d621d10-c1a9-45a7-82f3-612aaf04a51c None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:05.322434 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d621d10-c1a9-45a7-82f3-612aaf04a51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:05.338545 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf713e8-fe1d-4931-953a-181c7ce0883c None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:05.345256 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-1d621d10-c1a9-45a7-82f3-612aaf04a51c None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:05.346448 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-1d621d10-c1a9-45a7-82f3-612aaf04a51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:05.375466 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:05.400345 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:05.460520 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 226/461] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:13:05.487948 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf713e8-fe1d-4931-953a-181c7ce0883c None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:05.519496 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d621d10-c1a9-45a7-82f3-612aaf04a51c None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:05.563258 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba75a31-b83f-4055-aed7-29d09c426502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:05.563887 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba75a31-b83f-4055-aed7-29d09c426502 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:05.564162 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba75a31-b83f-4055-aed7-29d09c426502 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:05.564424 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba75a31-b83f-4055-aed7-29d09c426502 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:05.577916 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba75a31-b83f-4055-aed7-29d09c426502 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:05.579888 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba75a31-b83f-4055-aed7-29d09c426502 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:05.597705 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-9ba75a31-b83f-4055-aed7-29d09c426502 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:05.597705 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:05.597705 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:05.597705 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:05.597705 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:05.597705 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:05.597705 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:05.597705 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:05.597705 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:05.597705 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:05.597705 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:05.597705 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:05.597705 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:05.597705 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:05.597705 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:05.597705 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:05.598613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:05.598613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:05.598613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:05.598613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:05.598613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:05.598613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:05.598613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:05.598613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:05.598613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:05.598613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:05.598613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:05.598613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:05.598613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:05.598613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:05.598613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:05.598613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:05.598613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:05.599485 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:05.599485 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:05.599485 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:05.601218 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 236/462] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:05.616297 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba75a31-b83f-4055-aed7-29d09c426502 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:05.653216 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d621d10-c1a9-45a7-82f3-612aaf04a51c None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:05.689794 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d08f21-f1e2-436a-b265-441ccddaf033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:05.690424 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d08f21-f1e2-436a-b265-441ccddaf033 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:05.690702 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d08f21-f1e2-436a-b265-441ccddaf033 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:05.691004 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d08f21-f1e2-436a-b265-441ccddaf033 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:05.702421 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d08f21-f1e2-436a-b265-441ccddaf033 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:05.703986 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d08f21-f1e2-436a-b265-441ccddaf033 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:05.722088 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-b2d08f21-f1e2-436a-b265-441ccddaf033 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:05.722088 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:05.722088 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:05.722088 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:05.722088 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:05.722088 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:05.722088 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:05.722088 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:05.722088 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:05.722088 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:05.722088 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:05.722088 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:05.722088 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:05.722088 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:05.722088 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:05.722088 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:05.723043 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:05.723043 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:05.723043 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:05.723043 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:05.723043 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:05.723043 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:05.723043 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:05.723043 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:05.723043 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:05.723043 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:05.723043 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:05.723043 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:05.723043 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:05.723043 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:05.723043 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:05.723043 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:05.723043 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:05.723875 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:05.723875 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:05.723875 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:05.723875 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 227/463] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:13:05.724948 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29c25d-3f01-4e86-94a9-b4b8de2d5192 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:05.725226 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29c25d-3f01-4e86-94a9-b4b8de2d5192 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:05.725492 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29c25d-3f01-4e86-94a9-b4b8de2d5192 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:05.727068 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 228/464] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:05.732231 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29c25d-3f01-4e86-94a9-b4b8de2d5192 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:05.748412 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:05.753017 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:05.759600 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba75a31-b83f-4055-aed7-29d09c426502 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:05.799461 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e938f6d6-ce5a-418d-ab67-375688962c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:05.800176 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938f6d6-ce5a-418d-ab67-375688962c9a None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:05.800456 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938f6d6-ce5a-418d-ab67-375688962c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:05.800729 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938f6d6-ce5a-418d-ab67-375688962c9a None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:05.801370 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e938f6d6-ce5a-418d-ab67-375688962c9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:05.801675 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e938f6d6-ce5a-418d-ab67-375688962c9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:05.803107 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e938f6d6-ce5a-418d-ab67-375688962c9a None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:05.819877 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 237/465] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:13:05.819877 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de49e17-bfd1-4d1f-a817-e5fa01fbefd6 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:05.819877 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de49e17-bfd1-4d1f-a817-e5fa01fbefd6 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:05.819877 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de49e17-bfd1-4d1f-a817-e5fa01fbefd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:05.838621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-7de49e17-bfd1-4d1f-a817-e5fa01fbefd6 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:05.839850 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-7de49e17-bfd1-4d1f-a817-e5fa01fbefd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:05.858773 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29c25d-3f01-4e86-94a9-b4b8de2d5192 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:05.898176 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b7003-08f0-4300-8f49-14f0121f6810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:05.898793 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b7003-08f0-4300-8f49-14f0121f6810 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:05.899099 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b7003-08f0-4300-8f49-14f0121f6810 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:05.899376 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b7003-08f0-4300-8f49-14f0121f6810 None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:05.900023 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97b7003-08f0-4300-8f49-14f0121f6810 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:05.900359 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97b7003-08f0-4300-8f49-14f0121f6810 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:05.901731 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97b7003-08f0-4300-8f49-14f0121f6810 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:05.911917 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 238/466] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:13:05.915155 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de49e17-bfd1-4d1f-a817-e5fa01fbefd6 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:05.922554 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 229/467] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:05.922554 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b7003-08f0-4300-8f49-14f0121f6810 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:06.002582 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:06.003417 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:06.024401 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de49e17-bfd1-4d1f-a817-e5fa01fbefd6 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:06.041887 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b7003-08f0-4300-8f49-14f0121f6810 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:06.073205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c941a467-8e37-43a3-9c78-3242cfdf98f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:06.074665 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941a467-8e37-43a3-9c78-3242cfdf98f5 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:06.075426 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941a467-8e37-43a3-9c78-3242cfdf98f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:06.075877 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941a467-8e37-43a3-9c78-3242cfdf98f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:06.086276 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1de70a5-2876-4540-8f07-073f5cf48b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:06.087014 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de70a5-2876-4540-8f07-073f5cf48b32 None admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:06.087346 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de70a5-2876-4540-8f07-073f5cf48b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:06.088154 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de70a5-2876-4540-8f07-073f5cf48b32 None admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:06.088891 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1de70a5-2876-4540-8f07-073f5cf48b32 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:06.090254 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c941a467-8e37-43a3-9c78-3242cfdf98f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:06.090844 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1de70a5-2876-4540-8f07-073f5cf48b32 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:06.092124 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c941a467-8e37-43a3-9c78-3242cfdf98f5 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:06.110991 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-c941a467-8e37-43a3-9c78-3242cfdf98f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:06.110991 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:06.110991 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:06.110991 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:06.110991 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:06.110991 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:06.110991 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:06.110991 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:06.110991 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:06.110991 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:06.110991 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:06.110991 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:06.110991 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:06.110991 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:06.110991 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:06.110991 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:06.111958 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:06.111958 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:06.111958 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:06.111958 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:06.111958 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:06.111958 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:06.111958 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:06.111958 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:06.111958 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:06.111958 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:06.111958 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:06.111958 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:06.111958 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:06.111958 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:06.111958 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:06.111958 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:06.111958 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:06.112866 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:06.112866 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:06.112866 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:06.119779 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 230/468] 10.222.0.52 () {60 vars in 1132 bytes} [Sat Sep 24 14:13:05 2022] POST /identity/v3/users => generated 248 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:13:06.119779 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1de70a5-2876-4540-8f07-073f5cf48b32 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:06.120619 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 239/469] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:06.124207 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c941a467-8e37-43a3-9c78-3242cfdf98f5 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:06.263025 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1de70a5-2876-4540-8f07-073f5cf48b32 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:06.263908 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c941a467-8e37-43a3-9c78-3242cfdf98f5 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:06.294697 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:06.303048 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c80abe1-f1f1-47d7-835e-26d3af6aafbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:06.303864 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c80abe1-f1f1-47d7-835e-26d3af6aafbe None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:06.303970 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763bb52-fcf1-4640-a980-5281268143b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:06.304080 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c80abe1-f1f1-47d7-835e-26d3af6aafbe None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:06.304255 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c80abe1-f1f1-47d7-835e-26d3af6aafbe None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:06.304479 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763bb52-fcf1-4640-a980-5281268143b1 None admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:06.304934 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763bb52-fcf1-4640-a980-5281268143b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:06.305032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c80abe1-f1f1-47d7-835e-26d3af6aafbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:06.305189 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c80abe1-f1f1-47d7-835e-26d3af6aafbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:06.305276 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763bb52-fcf1-4640-a980-5281268143b1 None admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:06.305899 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763bb52-fcf1-4640-a980-5281268143b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:06.307120 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c80abe1-f1f1-47d7-835e-26d3af6aafbe None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:06.308239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763bb52-fcf1-4640-a980-5281268143b1 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:06.314627 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:06.320864 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 240/470] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:13:06.322204 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57485a-fe69-44c3-955e-e22e99490c6a None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:06.322483 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57485a-fe69-44c3-955e-e22e99490c6a None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:06.322753 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57485a-fe69-44c3-955e-e22e99490c6a None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:06.323964 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 241/471] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:06.324651 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57485a-fe69-44c3-955e-e22e99490c6a None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:06.339180 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 231/472] 10.222.0.52 () {60 vars in 1132 bytes} [Sat Sep 24 14:13:06 2022] POST /identity/v3/users => generated 246 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:13:06.339995 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763bb52-fcf1-4640-a980-5281268143b1 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:06.472977 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57485a-fe69-44c3-955e-e22e99490c6a None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:06.481937 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763bb52-fcf1-4640-a980-5281268143b1 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:06.523036 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ebde8-aa12-4643-b269-cbe5dc7e9e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:06.523803 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ebde8-aa12-4643-b269-cbe5dc7e9e91 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:06.524327 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ebde8-aa12-4643-b269-cbe5dc7e9e91 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:06.524607 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ebde8-aa12-4643-b269-cbe5dc7e9e91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:06.531194 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3deb91-9ce0-4ec0-a333-6f378ce27b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:06.531751 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3deb91-9ce0-4ec0-a333-6f378ce27b38 None admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:06.532083 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3deb91-9ce0-4ec0-a333-6f378ce27b38 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:06.532605 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3deb91-9ce0-4ec0-a333-6f378ce27b38 None admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:06.533844 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3deb91-9ce0-4ec0-a333-6f378ce27b38 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:06.535876 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3deb91-9ce0-4ec0-a333-6f378ce27b38 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:06.539456 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ebde8-aa12-4643-b269-cbe5dc7e9e91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:06.541265 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ebde8-aa12-4643-b269-cbe5dc7e9e91 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:06.560290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-5d7ebde8-aa12-4643-b269-cbe5dc7e9e91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:06.560290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:06.560290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:06.560290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:06.560290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:06.560290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:06.560290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:06.560290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:06.560290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:06.560290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:06.560290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:06.560290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:06.560290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:06.560290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:06.560290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:06.560290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:06.561234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:06.561234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:06.561234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:06.561234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:06.561234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:06.561234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:06.561234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:06.561234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:06.561234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:06.561234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:06.561234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:06.561234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:06.561234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:06.561234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:06.561234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:06.561234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:06.561234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:06.562079 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:06.562079 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:06.562079 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:06.562251 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 232/473] 10.222.0.52 () {60 vars in 1132 bytes} [Sat Sep 24 14:13:06 2022] POST /identity/v3/users => generated 248 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:13:06.562787 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60370dc-72d9-44e5-a6e3-9fef9788fd59 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:06.563104 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60370dc-72d9-44e5-a6e3-9fef9788fd59 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:06.563376 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60370dc-72d9-44e5-a6e3-9fef9788fd59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:06.564629 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 242/474] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:06.570635 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ebde8-aa12-4643-b269-cbe5dc7e9e91 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:06.586358 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-a60370dc-72d9-44e5-a6e3-9fef9788fd59 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:06.587674 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-a60370dc-72d9-44e5-a6e3-9fef9788fd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:06.632564 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:06.636531 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:06.684058 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 233/475] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:13:06.715596 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ebde8-aa12-4643-b269-cbe5dc7e9e91 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:06.764354 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c27bef5-8605-4720-bb89-1a686ff58ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:06.765326 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27bef5-8605-4720-bb89-1a686ff58ec4 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:06.765809 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27bef5-8605-4720-bb89-1a686ff58ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:06.766299 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27bef5-8605-4720-bb89-1a686ff58ec4 None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:06.767191 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c27bef5-8605-4720-bb89-1a686ff58ec4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:06.767700 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c27bef5-8605-4720-bb89-1a686ff58ec4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:06.769322 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c27bef5-8605-4720-bb89-1a686ff58ec4 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:06.785542 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 243/476] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:13:06.791542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60370dc-72d9-44e5-a6e3-9fef9788fd59 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:06.908503 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60370dc-72d9-44e5-a6e3-9fef9788fd59 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:06.945118 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc52d1e-f006-4aab-bd5f-04e4ef08dc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:06.946513 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc52d1e-f006-4aab-bd5f-04e4ef08dc3c None admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:06.947590 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc52d1e-f006-4aab-bd5f-04e4ef08dc3c None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:06.948168 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc52d1e-f006-4aab-bd5f-04e4ef08dc3c None admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:06.949024 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc52d1e-f006-4aab-bd5f-04e4ef08dc3c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:06.950659 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc52d1e-f006-4aab-bd5f-04e4ef08dc3c None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:06.971370 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c27bef5-8605-4720-bb89-1a686ff58ec4 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:06.985214 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 234/477] 10.222.0.52 () {60 vars in 1132 bytes} [Sat Sep 24 14:13:06 2022] POST /identity/v3/users => generated 249 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:13:07.098101 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c27bef5-8605-4720-bb89-1a686ff58ec4 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:07.107261 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:07.138329 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd2b36-af7b-421c-b196-79b39b83d53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:07.138963 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd2b36-af7b-421c-b196-79b39b83d53f None admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:07.139245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd2b36-af7b-421c-b196-79b39b83d53f None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:07.139514 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd2b36-af7b-421c-b196-79b39b83d53f None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:07.140278 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdd2b36-af7b-421c-b196-79b39b83d53f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:07.141682 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdd2b36-af7b-421c-b196-79b39b83d53f None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:07.151330 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.sql.core [None req-efdd2b36-af7b-421c-b196-79b39b83d53f None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 24 14:13:07.151330 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 24 14:13:07.151330 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [parameters: {'id': 'a06b1e1fe1974e0ea7172a0a87d8375e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 24 14:13:07.151330 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81496) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 24 14:13:07.168596 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-efdd2b36-af7b-421c-b196-79b39b83d53f None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 24 14:13:07.168596 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:07.168596 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 24 14:13:07.168596 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 24 14:13:07.168596 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 24 14:13:07.168596 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 24 14:13:07.168596 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 24 14:13:07.168596 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application result = self._query(query) Sep 24 14:13:07.168596 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 24 14:13:07.168596 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application conn.query(q) Sep 24 14:13:07.168596 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 24 14:13:07.168596 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 24 14:13:07.168596 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 24 14:13:07.168596 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application result.read() Sep 24 14:13:07.168596 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 24 14:13:07.169538 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 24 14:13:07.169538 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 24 14:13:07.169538 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 24 14:13:07.169538 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 24 14:13:07.169538 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 24 14:13:07.169538 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 24 14:13:07.169538 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 24 14:13:07.169538 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 24 14:13:07.169538 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:07.169538 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 24 14:13:07.169538 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:07.169538 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:07.169538 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 24 14:13:07.169538 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 24 14:13:07.169538 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 24 14:13:07.169538 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return ref.to_dict() Sep 24 14:13:07.169538 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 24 14:13:07.169538 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application next(self.gen) Sep 24 14:13:07.170454 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 24 14:13:07.170454 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application yield resource Sep 24 14:13:07.170454 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 24 14:13:07.170454 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application next(self.gen) Sep 24 14:13:07.170454 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 24 14:13:07.170454 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self.session.rollback() Sep 24 14:13:07.170454 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 24 14:13:07.170454 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self.force_reraise() Sep 24 14:13:07.170454 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 24 14:13:07.170454 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise self.value Sep 24 14:13:07.170454 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 24 14:13:07.170454 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 24 14:13:07.170454 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 24 14:13:07.170454 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application session.commit() Sep 24 14:13:07.170454 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 24 14:13:07.170454 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 24 14:13:07.170454 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 24 14:13:07.171835 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self._prepare_impl() Sep 24 14:13:07.171835 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 24 14:13:07.171835 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self.session.flush() Sep 24 14:13:07.171835 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 24 14:13:07.171835 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self._flush(objects) Sep 24 14:13:07.171835 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 24 14:13:07.171835 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 24 14:13:07.171835 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 24 14:13:07.171835 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application compat.raise_( Sep 24 14:13:07.171835 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 24 14:13:07.171835 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception Sep 24 14:13:07.171835 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 24 14:13:07.171835 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application flush_context.execute() Sep 24 14:13:07.171835 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 24 14:13:07.171835 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rec.execute(self) Sep 24 14:13:07.171835 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 24 14:13:07.171835 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 24 14:13:07.172703 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 24 14:13:07.172703 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 24 14:13:07.172703 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 24 14:13:07.172703 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 24 14:13:07.172703 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 24 14:13:07.172703 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 24 14:13:07.172703 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 24 14:13:07.172703 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 24 14:13:07.172703 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 24 14:13:07.172703 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 24 14:13:07.172703 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 24 14:13:07.172703 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 24 14:13:07.172703 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 24 14:13:07.172703 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 24 14:13:07.172703 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 24 14:13:07.172703 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception Sep 24 14:13:07.173514 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 24 14:13:07.173514 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 24 14:13:07.173514 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 24 14:13:07.173514 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 24 14:13:07.173514 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 24 14:13:07.173514 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application result = self._query(query) Sep 24 14:13:07.173514 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 24 14:13:07.173514 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application conn.query(q) Sep 24 14:13:07.173514 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 24 14:13:07.173514 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 24 14:13:07.173514 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 24 14:13:07.173514 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application result.read() Sep 24 14:13:07.173514 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 24 14:13:07.173514 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 24 14:13:07.173514 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 24 14:13:07.173514 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 24 14:13:07.174288 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 24 14:13:07.174288 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 24 14:13:07.174288 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 24 14:13:07.174288 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 24 14:13:07.174288 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 24 14:13:07.174288 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 24 14:13:07.174288 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [parameters: {'id': 'a06b1e1fe1974e0ea7172a0a87d8375e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 24 14:13:07.174288 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 24 14:13:07.174288 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:07.174288 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:13:07.174288 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:07.174288 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:07.174288 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:07.174288 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:07.174288 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:07.174288 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:07.175132 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:07.175132 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:07.175132 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:07.175132 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:07.175132 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:07.175132 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:07.175132 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 24 14:13:07.175132 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 24 14:13:07.175132 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:07.175132 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:07.175132 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 24 14:13:07.175132 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 24 14:13:07.175132 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 24 14:13:07.175132 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 24 14:13:07.175132 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 24 14:13:07.175132 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:07.176028 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 244/478] 10.222.0.52 () {60 vars in 1131 bytes} [Sat Sep 24 14:13:06 2022] POST /identity/v3/roles => generated 144 bytes in 199 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:13:07.177106 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc52d1e-f006-4aab-bd5f-04e4ef08dc3c None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:07.207763 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:07.294675 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc52d1e-f006-4aab-bd5f-04e4ef08dc3c None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:07.330118 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1d53e8-f76d-4a47-8fee-3d6224b4358a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:07.330754 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d53e8-f76d-4a47-8fee-3d6224b4358a None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:07.331061 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d53e8-f76d-4a47-8fee-3d6224b4358a None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:07.331332 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d53e8-f76d-4a47-8fee-3d6224b4358a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:07.342657 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1d53e8-f76d-4a47-8fee-3d6224b4358a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:13:07.348007 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-dd1d53e8-f76d-4a47-8fee-3d6224b4358a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:13:07.348007 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:07.348007 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:07.348007 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:07.348007 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:07.348007 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:07.348007 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:07.348007 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:07.348007 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:07.348007 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:07.348007 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:07.348007 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:07.348007 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:13:07.348007 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:13:07.348007 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:13:07.348007 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:13:07.350098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:13:07.350098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:13:07.350098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:13:07.350098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:13:07.350098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:07.350098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:07.350098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:07.350098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:07.350098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:07.350098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:07.350098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:07.350098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:07.350098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:07.350098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:07.350098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:07.350098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:07.352111 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:07.352111 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:07.352111 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:07.352111 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:07.352111 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:13:07.352111 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:13:07.352111 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:13:07.352111 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:13:07.352111 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:13:07.352111 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:13:07.352111 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 14:13:07.352111 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:07.352111 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 235/479] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 14:13:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:07.358939 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd2b36-af7b-421c-b196-79b39b83d53f None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:07.466197 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd2b36-af7b-421c-b196-79b39b83d53f None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:07.502252 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d017391-d1f1-45ab-b12e-46d86a0e52e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:07.502887 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d017391-d1f1-45ab-b12e-46d86a0e52e7 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:07.503168 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d017391-d1f1-45ab-b12e-46d86a0e52e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:07.503437 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d017391-d1f1-45ab-b12e-46d86a0e52e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:07.504159 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d017391-d1f1-45ab-b12e-46d86a0e52e7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:07.504464 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d017391-d1f1-45ab-b12e-46d86a0e52e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:07.505870 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d017391-d1f1-45ab-b12e-46d86a0e52e7 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:07.514543 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 245/480] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:13:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:08.804831 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e84c1-e532-40fc-806f-cf287aadc915 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:08.805346 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e84c1-e532-40fc-806f-cf287aadc915 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:08.805346 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e84c1-e532-40fc-806f-cf287aadc915 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:08.807575 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 236/481] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:08.812774 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982ab24-067e-4c41-8bcc-d501d6cb4a94 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:08.813183 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982ab24-067e-4c41-8bcc-d501d6cb4a94 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:08.813324 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982ab24-067e-4c41-8bcc-d501d6cb4a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:08.832645 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-6982ab24-067e-4c41-8bcc-d501d6cb4a94 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:08.834102 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-6982ab24-067e-4c41-8bcc-d501d6cb4a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:08.861297 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:08.907489 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 246/482] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:13:09.059545 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e9d17-4a2e-4401-a065-44beec8c2509 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:09.059782 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e9d17-4a2e-4401-a065-44beec8c2509 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:09.060168 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e9d17-4a2e-4401-a065-44beec8c2509 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:09.061718 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 237/483] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:09.066836 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf09fd-7f7d-4728-b7fb-fa8899e01074 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:09.067125 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf09fd-7f7d-4728-b7fb-fa8899e01074 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:09.067396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf09fd-7f7d-4728-b7fb-fa8899e01074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:09.090774 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-0abf09fd-7f7d-4728-b7fb-fa8899e01074 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:09.092076 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-0abf09fd-7f7d-4728-b7fb-fa8899e01074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:09.164871 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 247/484] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:13:09.256266 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e9d17-4a2e-4401-a065-44beec8c2509 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:09.360199 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:09.384760 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e9d17-4a2e-4401-a065-44beec8c2509 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:09.424555 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f18a3-2578-4084-af00-7f5afeef25e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:09.425610 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f18a3-2578-4084-af00-7f5afeef25e7 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:09.426087 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f18a3-2578-4084-af00-7f5afeef25e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:09.426620 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f18a3-2578-4084-af00-7f5afeef25e7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:09.437831 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92f18a3-2578-4084-af00-7f5afeef25e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:13:09.443450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-e92f18a3-2578-4084-af00-7f5afeef25e7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:13:09.443450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:09.443450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:09.443450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:09.443450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:09.443450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:09.443450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:09.443450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:09.443450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:09.443450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:09.443450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:09.443450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:09.443450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:13:09.443450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:13:09.443450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:13:09.443450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:13:09.444305 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:13:09.444305 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:13:09.444305 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:13:09.444305 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:13:09.444305 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:09.444305 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:09.444305 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:09.444305 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:09.444305 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:09.444305 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:09.444305 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:09.444305 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:09.444305 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:09.444305 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:09.444305 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:09.444305 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:09.445063 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:09.445063 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:09.445063 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:09.445063 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:09.445063 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:13:09.445063 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:13:09.445063 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:13:09.445063 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:13:09.445063 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:13:09.445063 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:13:09.445063 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:13:09.445063 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:09.447375 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 238/485] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 14:13:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:09.453369 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abf09fd-7f7d-4728-b7fb-fa8899e01074 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:09.493349 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8841cb-80a8-40e7-a9d2-9f51aa09d041 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:09.494520 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8841cb-80a8-40e7-a9d2-9f51aa09d041 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:09.495037 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8841cb-80a8-40e7-a9d2-9f51aa09d041 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:09.496868 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 239/486] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:13:09.510951 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3bb004-d6e7-4eb9-8a03-24122b32dd14 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:09.511499 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3bb004-d6e7-4eb9-8a03-24122b32dd14 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:09.512010 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3bb004-d6e7-4eb9-8a03-24122b32dd14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:09.546256 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-4d3bb004-d6e7-4eb9-8a03-24122b32dd14 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:09.547987 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-4d3bb004-d6e7-4eb9-8a03-24122b32dd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:09.594224 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:09.608772 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abf09fd-7f7d-4728-b7fb-fa8899e01074 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:09.666955 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f2bd2-2a75-4c3b-b855-05d47da819f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:09.667558 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 240/487] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:13:09.668588 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3bb004-d6e7-4eb9-8a03-24122b32dd14 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:09.670938 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f2bd2-2a75-4c3b-b855-05d47da819f2 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:09.671453 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f2bd2-2a75-4c3b-b855-05d47da819f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:09.671926 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f2bd2-2a75-4c3b-b855-05d47da819f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:09.672818 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f2bd2-2a75-4c3b-b855-05d47da819f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:09.674057 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f2bd2-2a75-4c3b-b855-05d47da819f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:09.675703 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f2bd2-2a75-4c3b-b855-05d47da819f2 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:09.688934 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 248/488] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 14:13:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:09.694377 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f2bd2-2a75-4c3b-b855-05d47da819f2 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:09.819642 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:09.851260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f2bd2-2a75-4c3b-b855-05d47da819f2 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:09.870852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3bb004-d6e7-4eb9-8a03-24122b32dd14 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:09.927981 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa31651-25b6-4e76-89f5-f29412368889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:09.928549 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa31651-25b6-4e76-89f5-f29412368889 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:09.928873 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa31651-25b6-4e76-89f5-f29412368889 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:09.929150 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa31651-25b6-4e76-89f5-f29412368889 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:09.941046 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa31651-25b6-4e76-89f5-f29412368889 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:13:09.944139 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a823c-bc10-42af-b7dc-77ffa7d85819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:09.944793 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a823c-bc10-42af-b7dc-77ffa7d85819 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:09.945074 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a823c-bc10-42af-b7dc-77ffa7d85819 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:09.945349 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a823c-bc10-42af-b7dc-77ffa7d85819 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:09.947794 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-caa31651-25b6-4e76-89f5-f29412368889 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:13:09.947794 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:09.947794 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:09.947794 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:09.947794 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:09.947794 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:09.947794 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:09.947794 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:09.947794 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:09.947794 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:09.947794 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:09.947794 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:09.947794 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:13:09.947794 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:13:09.947794 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:13:09.947794 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:13:09.948655 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:13:09.948655 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:13:09.948655 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:13:09.948655 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:13:09.948655 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:09.948655 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:09.948655 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:09.948655 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:09.948655 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:09.948655 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:09.948655 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:09.948655 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:09.948655 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:09.948655 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:09.948655 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:09.948655 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:09.949428 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:09.949428 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:09.949428 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:09.949428 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:09.949428 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:13:09.949428 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:13:09.949428 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:13:09.949428 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:13:09.949428 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:13:09.949428 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:13:09.949428 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:13:09.949428 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:09.950107 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 241/489] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 14:13:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:09.955098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa31651-25b6-4e76-89f5-f29412368889 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:09.983674 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7a823c-bc10-42af-b7dc-77ffa7d85819 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:09.985145 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7a823c-bc10-42af-b7dc-77ffa7d85819 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:10.012627 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-cf7a823c-bc10-42af-b7dc-77ffa7d85819 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:10.012627 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:10.012627 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:10.012627 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:10.012627 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:10.012627 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:10.012627 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:10.012627 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:10.012627 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:10.012627 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:10.012627 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:10.012627 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:10.012627 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:10.012627 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:10.012627 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:10.012627 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:10.014795 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:10.014795 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:10.014795 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:10.014795 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:10.014795 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:10.014795 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:10.014795 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:10.014795 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:10.014795 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:10.014795 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:10.014795 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:10.014795 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:10.014795 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:10.014795 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:10.014795 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:10.014795 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:10.014795 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:10.016888 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:10.016888 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:10.016888 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:10.016888 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 249/490] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:13:10.016888 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a823c-bc10-42af-b7dc-77ffa7d85819 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:10.028276 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:10.076665 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa31651-25b6-4e76-89f5-f29412368889 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:10.110953 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62afc4b0-7a72-47bb-a2aa-8dbc1ebf072f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:10.111560 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afc4b0-7a72-47bb-a2aa-8dbc1ebf072f None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:10.111842 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afc4b0-7a72-47bb-a2aa-8dbc1ebf072f None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:10.112113 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afc4b0-7a72-47bb-a2aa-8dbc1ebf072f None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:10.112780 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62afc4b0-7a72-47bb-a2aa-8dbc1ebf072f None admin] RBAC: Adding query filter params (name=service) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:10.113082 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62afc4b0-7a72-47bb-a2aa-8dbc1ebf072f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:10.114455 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62afc4b0-7a72-47bb-a2aa-8dbc1ebf072f None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:10.116724 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:10.124353 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 242/491] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 14:13:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:10.125148 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62afc4b0-7a72-47bb-a2aa-8dbc1ebf072f None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:10.134184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a823c-bc10-42af-b7dc-77ffa7d85819 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:10.169639 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a941f1-5ba7-47ec-9e53-b21e1443ff7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:10.170552 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a941f1-5ba7-47ec-9e53-b21e1443ff7b None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:10.171036 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a941f1-5ba7-47ec-9e53-b21e1443ff7b None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:10.171493 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a941f1-5ba7-47ec-9e53-b21e1443ff7b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:10.183293 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a941f1-5ba7-47ec-9e53-b21e1443ff7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:13:10.188460 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-f6a941f1-5ba7-47ec-9e53-b21e1443ff7b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:13:10.188460 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:10.188460 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:10.188460 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:10.188460 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:10.188460 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:10.188460 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:10.188460 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:10.188460 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:10.188460 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:10.188460 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:10.188460 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:10.188460 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:13:10.188460 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:13:10.188460 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:13:10.188460 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:13:10.189293 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:13:10.189293 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:13:10.189293 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:13:10.189293 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:13:10.189293 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:10.189293 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:10.189293 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:10.189293 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:10.189293 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:10.189293 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:10.189293 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:10.189293 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:10.189293 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:10.189293 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:10.189293 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:10.189293 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:10.190067 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:10.190067 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:10.190067 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:10.190067 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:10.190067 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:13:10.190067 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:13:10.190067 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:13:10.190067 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:13:10.190067 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:13:10.190067 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:13:10.190067 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:13:10.190067 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:10.190067 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 250/492] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 14:13:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:10.191566 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f9039-25bf-4f66-a4ad-494e1e278aa2 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:10.191851 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f9039-25bf-4f66-a4ad-494e1e278aa2 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:10.192238 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f9039-25bf-4f66-a4ad-494e1e278aa2 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:10.198173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 251/493] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:10.198173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f9039-25bf-4f66-a4ad-494e1e278aa2 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:10.236429 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62afc4b0-7a72-47bb-a2aa-8dbc1ebf072f None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:10.267533 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:10.271117 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473462c-bf56-4eca-b331-ae736eb4f407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:10.271728 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473462c-bf56-4eca-b331-ae736eb4f407 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:10.272020 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473462c-bf56-4eca-b331-ae736eb4f407 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:10.272284 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473462c-bf56-4eca-b331-ae736eb4f407 None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:10.272939 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7473462c-bf56-4eca-b331-ae736eb4f407 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:10.273241 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7473462c-bf56-4eca-b331-ae736eb4f407 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:10.274602 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7473462c-bf56-4eca-b331-ae736eb4f407 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:10.287621 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 243/494] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:10.288709 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473462c-bf56-4eca-b331-ae736eb4f407 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:10.314993 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f9039-25bf-4f66-a4ad-494e1e278aa2 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:10.351489 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f80ccb6-c482-4afd-bd5d-8a82d82c16d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:10.352121 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f80ccb6-c482-4afd-bd5d-8a82d82c16d0 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:10.352411 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f80ccb6-c482-4afd-bd5d-8a82d82c16d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:10.352688 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f80ccb6-c482-4afd-bd5d-8a82d82c16d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:10.355955 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:10.363722 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f80ccb6-c482-4afd-bd5d-8a82d82c16d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:10.365185 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f80ccb6-c482-4afd-bd5d-8a82d82c16d0 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:10.379776 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-5f80ccb6-c482-4afd-bd5d-8a82d82c16d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:10.379776 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:10.379776 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:10.379776 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:10.379776 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:10.379776 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:10.379776 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:10.379776 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:10.379776 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:10.379776 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:10.379776 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:10.379776 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:10.379776 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:10.379776 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:10.379776 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:10.379776 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:10.380660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:10.380660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:10.380660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:10.380660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:10.380660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:10.380660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:10.380660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:10.380660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:10.380660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:10.380660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:10.380660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:10.380660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:10.380660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:10.380660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:10.380660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:10.380660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:10.380660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:10.381474 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:10.381474 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:10.381474 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:10.381474 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 252/495] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:10.382301 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f67d55-b4d0-40c8-9508-11a2f828eb5f None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:10.382301 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f67d55-b4d0-40c8-9508-11a2f828eb5f None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:10.382525 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f67d55-b4d0-40c8-9508-11a2f828eb5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:10.401495 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-30f67d55-b4d0-40c8-9508-11a2f828eb5f None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:10.403029 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-30f67d55-b4d0-40c8-9508-11a2f828eb5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:10.405922 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473462c-bf56-4eca-b331-ae736eb4f407 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:10.441540 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f79bc0-994b-456b-8054-344dd0e03afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:10.442071 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f79bc0-994b-456b-8054-344dd0e03afe None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:10.442349 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f79bc0-994b-456b-8054-344dd0e03afe None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:10.442620 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f79bc0-994b-456b-8054-344dd0e03afe None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:10.443318 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f79bc0-994b-456b-8054-344dd0e03afe None admin] RBAC: Adding query filter params (name=service) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:10.443621 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f79bc0-994b-456b-8054-344dd0e03afe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:10.444974 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f79bc0-994b-456b-8054-344dd0e03afe None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:10.452827 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 244/496] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 14:13:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:10.453599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f79bc0-994b-456b-8054-344dd0e03afe None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:10.471972 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 253/497] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:13:10.472963 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f67d55-b4d0-40c8-9508-11a2f828eb5f None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:10.533626 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:10.562421 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f79bc0-994b-456b-8054-344dd0e03afe None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:10.582163 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:10.590083 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f67d55-b4d0-40c8-9508-11a2f828eb5f None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:10.597016 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225f3ddb-2563-4ac7-b67e-9771df73e921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:10.597629 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f3ddb-2563-4ac7-b67e-9771df73e921 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:10.597903 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f3ddb-2563-4ac7-b67e-9771df73e921 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:10.598174 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f3ddb-2563-4ac7-b67e-9771df73e921 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:10.606190 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225f3ddb-2563-4ac7-b67e-9771df73e921 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:10.607645 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225f3ddb-2563-4ac7-b67e-9771df73e921 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:10.620438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-225f3ddb-2563-4ac7-b67e-9771df73e921 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 24 14:13:10.620438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:10.620438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:10.620438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:10.620438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:10.620438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:10.620438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:10.620438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:10.620438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:10.620438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:10.620438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:10.620438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:10.620438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:13:10.620438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:13:10.620438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:13:10.620438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:13:10.621319 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:10.621319 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:10.621319 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:13:10.621319 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:10.621319 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:13:10.621319 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:10.621319 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:10.621319 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:10.621319 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:10.621319 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:10.621319 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:10.621319 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:10.621319 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:10.621319 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:10.621319 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:10.621319 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:10.621319 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:10.621319 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:10.622172 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:10.622172 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:10.622172 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:13:10.622172 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:13:10.622172 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:13:10.622172 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:13:10.622172 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:13:10.622172 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:13:10.622172 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 24 14:13:10.622172 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:10.622172 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 245/498] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:13:10 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:10.623192 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e458f3e-450c-4a60-8cb6-e2d3d40476a6 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:10.623192 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e458f3e-450c-4a60-8cb6-e2d3d40476a6 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:10.623192 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e458f3e-450c-4a60-8cb6-e2d3d40476a6 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:10.624601 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 246/499] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:10.629114 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e458f3e-450c-4a60-8cb6-e2d3d40476a6 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:10.630995 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83dc5e-a31f-49bb-883c-f539e0850430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:10.631794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83dc5e-a31f-49bb-883c-f539e0850430 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:10.632236 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83dc5e-a31f-49bb-883c-f539e0850430 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:10.632703 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83dc5e-a31f-49bb-883c-f539e0850430 None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:10.633349 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d83dc5e-a31f-49bb-883c-f539e0850430 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:10.633648 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d83dc5e-a31f-49bb-883c-f539e0850430 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:10.635641 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d83dc5e-a31f-49bb-883c-f539e0850430 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:10.651816 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 254/500] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:10.651816 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83dc5e-a31f-49bb-883c-f539e0850430 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:10.743069 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e458f3e-450c-4a60-8cb6-e2d3d40476a6 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:10.759964 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83dc5e-a31f-49bb-883c-f539e0850430 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:10.768794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:10.783332 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bffc607-856d-4ce9-b4ce-59790a96d1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:10.783960 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffc607-856d-4ce9-b4ce-59790a96d1a0 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:10.784237 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffc607-856d-4ce9-b4ce-59790a96d1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:10.784507 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffc607-856d-4ce9-b4ce-59790a96d1a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:10.795028 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bffc607-856d-4ce9-b4ce-59790a96d1a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:10.795597 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff023d3-c537-4c0a-bced-03b7f8219545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:10.796204 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff023d3-c537-4c0a-bced-03b7f8219545 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:10.796497 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bffc607-856d-4ce9-b4ce-59790a96d1a0 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:10.796588 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff023d3-c537-4c0a-bced-03b7f8219545 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:10.796770 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff023d3-c537-4c0a-bced-03b7f8219545 None admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:10.797487 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff023d3-c537-4c0a-bced-03b7f8219545 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:10.797790 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff023d3-c537-4c0a-bced-03b7f8219545 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:10.799251 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff023d3-c537-4c0a-bced-03b7f8219545 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:10.810474 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-9bffc607-856d-4ce9-b4ce-59790a96d1a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:10.810474 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:10.810474 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:10.810474 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:10.810474 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:10.810474 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:10.810474 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:10.810474 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:10.810474 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:10.810474 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:10.810474 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:10.810474 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:10.810474 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:10.810474 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:10.810474 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:10.810474 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:10.811450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:10.811450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:10.811450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:10.811450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:10.811450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:10.811450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:10.811450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:10.811450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:10.811450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:10.811450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:10.811450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:10.811450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:10.811450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:10.811450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:10.811450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:10.811450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:10.811450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:10.812259 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:10.812259 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:10.812259 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:10.812259 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 247/501] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:10.812867 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955be79-57d4-4b87-9caa-e63d447d7fac None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:10.813022 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955be79-57d4-4b87-9caa-e63d447d7fac None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:10.813270 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955be79-57d4-4b87-9caa-e63d447d7fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:10.833680 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-6955be79-57d4-4b87-9caa-e63d447d7fac None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:10.834910 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-6955be79-57d4-4b87-9caa-e63d447d7fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:10.836429 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 255/502] 10.222.0.52 () {56 vars in 1137 bytes} [Sat Sep 24 14:13:10 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:10.837265 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff023d3-c537-4c0a-bced-03b7f8219545 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:10.862467 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:10.911499 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 248/503] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:13:10.913461 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6955be79-57d4-4b87-9caa-e63d447d7fac None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:10.945407 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff023d3-c537-4c0a-bced-03b7f8219545 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:10.982101 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225357f-643e-4714-a502-149049796366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:10.982729 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225357f-643e-4714-a502-149049796366 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:10.983041 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225357f-643e-4714-a502-149049796366 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:10.983309 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225357f-643e-4714-a502-149049796366 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:10.990341 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e225357f-643e-4714-a502-149049796366 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:10.991826 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e225357f-643e-4714-a502-149049796366 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:11.006206 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-e225357f-643e-4714-a502-149049796366 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 14:13:11.006206 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:11.006206 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:11.006206 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:11.006206 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:11.006206 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:11.006206 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:11.006206 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:11.006206 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:11.006206 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:11.006206 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:11.006206 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:11.006206 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:13:11.006206 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:13:11.006206 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:13:11.006206 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:13:11.007302 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:11.007302 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:11.007302 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:13:11.007302 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:11.007302 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:13:11.007302 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:11.007302 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:11.007302 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:11.007302 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:11.007302 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:11.007302 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:11.007302 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:11.007302 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:11.007302 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:11.007302 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:11.007302 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:11.007302 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:11.007302 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:11.008174 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:11.008174 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:11.008174 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:13:11.008174 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:13:11.008174 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:13:11.008174 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:13:11.008174 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:13:11.008174 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:13:11.008174 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 14:13:11.008174 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:11.008174 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 256/504] 10.222.0.52 () {56 vars in 1093 bytes} [Sat Sep 24 14:13:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:11.008174 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225357f-643e-4714-a502-149049796366 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:11.026690 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6955be79-57d4-4b87-9caa-e63d447d7fac None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:11.042623 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:11.062498 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0277c8-fd1b-4ea9-a02b-5871c0dc07aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:11.063065 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0277c8-fd1b-4ea9-a02b-5871c0dc07aa None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:11.063342 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0277c8-fd1b-4ea9-a02b-5871c0dc07aa None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:11.063920 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0277c8-fd1b-4ea9-a02b-5871c0dc07aa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:11.075175 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0277c8-fd1b-4ea9-a02b-5871c0dc07aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:13:11.080488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-ff0277c8-fd1b-4ea9-a02b-5871c0dc07aa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:13:11.080488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:11.080488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:11.080488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:11.080488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:11.080488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:11.080488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:11.080488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:11.080488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:11.080488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:11.080488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:11.080488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:11.080488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:13:11.080488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:13:11.080488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:13:11.080488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:13:11.081347 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:13:11.081347 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:13:11.081347 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:13:11.081347 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:13:11.081347 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:11.081347 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:11.081347 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:11.081347 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:11.081347 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:11.081347 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:11.081347 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:11.081347 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:11.081347 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:11.081347 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:11.081347 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:11.081347 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:11.082116 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:11.082116 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:11.082116 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:11.082116 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:11.082116 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:13:11.082116 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:13:11.082116 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:13:11.082116 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:13:11.082116 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:13:11.082116 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:13:11.082116 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:13:11.082116 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:11.082116 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 249/505] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 14:13:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:11.082747 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0277c8-fd1b-4ea9-a02b-5871c0dc07aa None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:11.111646 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:11.125180 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225357f-643e-4714-a502-149049796366 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:11.159808 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477cf943-82d5-43e6-81c3-357f05ed55a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:11.160422 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477cf943-82d5-43e6-81c3-357f05ed55a6 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:11.160707 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477cf943-82d5-43e6-81c3-357f05ed55a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:11.160995 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477cf943-82d5-43e6-81c3-357f05ed55a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:11.161629 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477cf943-82d5-43e6-81c3-357f05ed55a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:11.161937 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477cf943-82d5-43e6-81c3-357f05ed55a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:11.163368 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477cf943-82d5-43e6-81c3-357f05ed55a6 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:11.175086 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 257/506] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:11.175893 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477cf943-82d5-43e6-81c3-357f05ed55a6 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:11.201330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0277c8-fd1b-4ea9-a02b-5871c0dc07aa None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:11.236803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef01d7-e089-43f6-8048-0d49d9599cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:11.237405 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef01d7-e089-43f6-8048-0d49d9599cdc None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:11.237683 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef01d7-e089-43f6-8048-0d49d9599cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:11.237948 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef01d7-e089-43f6-8048-0d49d9599cdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:11.282711 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:11.298047 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477cf943-82d5-43e6-81c3-357f05ed55a6 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:11.308147 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ef01d7-e089-43f6-8048-0d49d9599cdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:11.309636 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ef01d7-e089-43f6-8048-0d49d9599cdc None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:11.330797 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0272644-6a92-4828-8930-6ac70efa7a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:11.331423 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0272644-6a92-4828-8930-6ac70efa7a0f None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:11.331705 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0272644-6a92-4828-8930-6ac70efa7a0f None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:11.331837 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:11.331974 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0272644-6a92-4828-8930-6ac70efa7a0f None admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:11.332685 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0272644-6a92-4828-8930-6ac70efa7a0f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:11.332986 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0272644-6a92-4828-8930-6ac70efa7a0f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:11.334535 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0272644-6a92-4828-8930-6ac70efa7a0f None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:11.352614 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 258/507] 10.222.0.52 () {56 vars in 1133 bytes} [Sat Sep 24 14:13:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:11.354041 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0272644-6a92-4828-8930-6ac70efa7a0f None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:11.411461 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 250/508] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:13:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:13:11.411461 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef01d7-e089-43f6-8048-0d49d9599cdc None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:11.455819 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0272644-6a92-4828-8930-6ac70efa7a0f None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:11.488814 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71983c1a-c07a-4a55-876b-6b1795970504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:11.489414 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71983c1a-c07a-4a55-876b-6b1795970504 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:11.489691 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71983c1a-c07a-4a55-876b-6b1795970504 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:11.489959 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71983c1a-c07a-4a55-876b-6b1795970504 None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:11.490612 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71983c1a-c07a-4a55-876b-6b1795970504 None admin] RBAC: Adding query filter params (name=service) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:11.490946 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71983c1a-c07a-4a55-876b-6b1795970504 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:11.492313 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71983c1a-c07a-4a55-876b-6b1795970504 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:11.501380 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 259/509] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 14:13:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:11.502184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71983c1a-c07a-4a55-876b-6b1795970504 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:11.509743 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef01d7-e089-43f6-8048-0d49d9599cdc None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:11.510891 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:11.532186 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:11.544449 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ba6d49-e214-4ce3-a6b0-affe22b6b82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:11.545013 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ba6d49-e214-4ce3-a6b0-affe22b6b82e None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:11.545286 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ba6d49-e214-4ce3-a6b0-affe22b6b82e None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:11.545555 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ba6d49-e214-4ce3-a6b0-affe22b6b82e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:11.561071 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ba6d49-e214-4ce3-a6b0-affe22b6b82e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:11.562514 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ba6d49-e214-4ce3-a6b0-affe22b6b82e None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:11.581217 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-a6ba6d49-e214-4ce3-a6b0-affe22b6b82e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:11.581217 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:11.581217 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:11.581217 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:11.581217 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:11.581217 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:11.581217 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:11.581217 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:11.581217 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:11.581217 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:11.581217 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:11.581217 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:11.581217 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:11.581217 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:11.581217 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:11.581217 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:11.582220 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:11.582220 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:11.582220 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:11.582220 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:11.582220 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:11.582220 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:11.582220 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:11.582220 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:11.582220 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:11.582220 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:11.582220 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:11.582220 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:11.582220 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:11.582220 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:11.582220 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:11.582220 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:11.582220 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:11.583398 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:11.583398 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:11.583398 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:11.583398 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 251/510] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:11.583398 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ba6d49-e214-4ce3-a6b0-affe22b6b82e None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:11.608830 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71983c1a-c07a-4a55-876b-6b1795970504 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:11.642492 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067647eb-a842-47f8-bbaa-c55819333552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:11.643148 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067647eb-a842-47f8-bbaa-c55819333552 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:11.643427 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067647eb-a842-47f8-bbaa-c55819333552 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:11.643698 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067647eb-a842-47f8-bbaa-c55819333552 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:11.651050 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067647eb-a842-47f8-bbaa-c55819333552 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:11.652505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067647eb-a842-47f8-bbaa-c55819333552 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:11.663704 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-067647eb-a842-47f8-bbaa-c55819333552 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 24 14:13:11.663704 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:11.663704 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:11.663704 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:11.663704 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:11.663704 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:11.663704 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:11.663704 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:11.663704 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:11.663704 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:11.663704 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:11.663704 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:11.663704 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:13:11.663704 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:13:11.663704 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:13:11.663704 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:13:11.664661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:11.664661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:11.664661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:13:11.664661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:11.664661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:13:11.664661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:11.664661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:11.664661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:11.664661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:11.664661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:11.664661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:11.664661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:11.664661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:11.664661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:11.664661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:11.664661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:11.664661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:11.664661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:11.665533 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:11.665533 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:11.665533 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:13:11.665533 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:13:11.665533 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:13:11.665533 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:13:11.665533 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:13:11.665533 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:13:11.665533 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 24 14:13:11.665533 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:11.665533 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 260/511] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:13:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:11.665533 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067647eb-a842-47f8-bbaa-c55819333552 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:11.686314 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ba6d49-e214-4ce3-a6b0-affe22b6b82e None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:11.720978 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1446ca8-cf19-4bf8-a952-ab70d36e80c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:11.721578 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1446ca8-cf19-4bf8-a952-ab70d36e80c3 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:11.721853 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1446ca8-cf19-4bf8-a952-ab70d36e80c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:11.722123 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1446ca8-cf19-4bf8-a952-ab70d36e80c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:11.732730 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:11.765308 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:11.767735 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067647eb-a842-47f8-bbaa-c55819333552 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:11.790360 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1446ca8-cf19-4bf8-a952-ab70d36e80c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:11.791938 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1446ca8-cf19-4bf8-a952-ab70d36e80c3 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:11.800972 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf6fdb-48a5-4a6d-bb4e-6a82c5c37e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:11.801535 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf6fdb-48a5-4a6d-bb4e-6a82c5c37e51 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:11.801812 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf6fdb-48a5-4a6d-bb4e-6a82c5c37e51 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:11.802081 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf6fdb-48a5-4a6d-bb4e-6a82c5c37e51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:11.812856 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccf6fdb-48a5-4a6d-bb4e-6a82c5c37e51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:11.814296 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccf6fdb-48a5-4a6d-bb4e-6a82c5c37e51 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:11.828840 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-cccf6fdb-48a5-4a6d-bb4e-6a82c5c37e51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:11.828840 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:11.828840 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:11.828840 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:11.828840 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:11.828840 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:11.828840 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:11.828840 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:11.828840 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:11.828840 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:11.828840 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:11.828840 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:11.828840 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:11.828840 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:11.828840 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:11.828840 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:11.829918 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:11.829918 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:11.829918 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:11.829918 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:11.829918 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:11.829918 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:11.829918 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:11.829918 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:11.829918 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:11.829918 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:11.829918 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:11.829918 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:11.829918 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:11.829918 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:11.829918 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:11.829918 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:11.829918 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:11.831001 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:11.831001 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:11.831001 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:11.831001 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 261/512] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:11.831001 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf6fdb-48a5-4a6d-bb4e-6a82c5c37e51 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:11.898173 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 252/513] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:13:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:13:11.898806 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1446ca8-cf19-4bf8-a952-ab70d36e80c3 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:11.933433 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf6fdb-48a5-4a6d-bb4e-6a82c5c37e51 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:11.950555 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:11.967339 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c00790-3a5c-45eb-bdf4-9e7df18b9dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:11.967962 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c00790-3a5c-45eb-bdf4-9e7df18b9dea None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:11.968245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c00790-3a5c-45eb-bdf4-9e7df18b9dea None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:11.968521 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c00790-3a5c-45eb-bdf4-9e7df18b9dea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:11.973407 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:11.978910 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c00790-3a5c-45eb-bdf4-9e7df18b9dea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:11.980357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c00790-3a5c-45eb-bdf4-9e7df18b9dea None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:11.994651 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-f0c00790-3a5c-45eb-bdf4-9e7df18b9dea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:11.994651 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:11.994651 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:11.994651 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:11.994651 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:11.994651 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:11.994651 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:11.994651 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:11.994651 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:11.994651 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:11.994651 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:11.994651 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:11.994651 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:11.994651 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:11.994651 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:11.994651 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:11.995742 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:11.995742 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:11.995742 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:11.995742 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:11.995742 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:11.995742 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:11.995742 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:11.995742 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:11.995742 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:11.995742 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:11.995742 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:11.995742 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:11.995742 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:11.995742 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:11.995742 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:11.995742 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:11.995742 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:11.996583 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:11.996583 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:11.996583 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:11.996583 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 262/514] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:11.996583 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c00790-3a5c-45eb-bdf4-9e7df18b9dea None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:12.003332 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1446ca8-cf19-4bf8-a952-ab70d36e80c3 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:12.037619 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e70d91a-2f42-4b7f-8ac9-0ec1132f733f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:12.038217 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70d91a-2f42-4b7f-8ac9-0ec1132f733f None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:12.038486 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70d91a-2f42-4b7f-8ac9-0ec1132f733f None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:12.038752 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70d91a-2f42-4b7f-8ac9-0ec1132f733f None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:12.039422 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e70d91a-2f42-4b7f-8ac9-0ec1132f733f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:12.039716 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e70d91a-2f42-4b7f-8ac9-0ec1132f733f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:12.041100 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e70d91a-2f42-4b7f-8ac9-0ec1132f733f None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:12.052335 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 253/515] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:12.053074 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e70d91a-2f42-4b7f-8ac9-0ec1132f733f None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:12.097072 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c00790-3a5c-45eb-bdf4-9e7df18b9dea None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:12.129635 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89e2d4f-6ec4-41c0-bef0-9574241f7e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:12.130238 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e2d4f-6ec4-41c0-bef0-9574241f7e11 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:12.130515 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e2d4f-6ec4-41c0-bef0-9574241f7e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:12.130789 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e2d4f-6ec4-41c0-bef0-9574241f7e11 None admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:12.131548 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89e2d4f-6ec4-41c0-bef0-9574241f7e11 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:12.131853 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89e2d4f-6ec4-41c0-bef0-9574241f7e11 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:12.133291 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89e2d4f-6ec4-41c0-bef0-9574241f7e11 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:12.150371 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 263/516] 10.222.0.52 () {56 vars in 1137 bytes} [Sat Sep 24 14:13:11 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:12.151198 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89e2d4f-6ec4-41c0-bef0-9574241f7e11 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:12.152451 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e70d91a-2f42-4b7f-8ac9-0ec1132f733f None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:12.182059 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:12.186317 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38daa21b-a4dd-4956-911b-77d9941dba6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:12.186976 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38daa21b-a4dd-4956-911b-77d9941dba6d None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:12.187259 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38daa21b-a4dd-4956-911b-77d9941dba6d None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:12.187531 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38daa21b-a4dd-4956-911b-77d9941dba6d None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:12.188167 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38daa21b-a4dd-4956-911b-77d9941dba6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:12.188469 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38daa21b-a4dd-4956-911b-77d9941dba6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:12.189829 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38daa21b-a4dd-4956-911b-77d9941dba6d None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:12.191252 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:12.202238 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 254/517] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:12.203074 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38daa21b-a4dd-4956-911b-77d9941dba6d None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:12.254023 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89e2d4f-6ec4-41c0-bef0-9574241f7e11 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:12.286855 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb050e56-82af-46d0-9910-3e734b17d661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:12.287480 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb050e56-82af-46d0-9910-3e734b17d661 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:12.287750 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb050e56-82af-46d0-9910-3e734b17d661 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:12.288027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb050e56-82af-46d0-9910-3e734b17d661 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:12.298237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb050e56-82af-46d0-9910-3e734b17d661 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:12.299719 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb050e56-82af-46d0-9910-3e734b17d661 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:12.299816 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38daa21b-a4dd-4956-911b-77d9941dba6d None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:12.315278 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-fb050e56-82af-46d0-9910-3e734b17d661 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:12.315278 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:12.315278 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:12.315278 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:12.315278 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:12.315278 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:12.315278 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:12.315278 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:12.315278 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:12.315278 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:12.315278 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:12.315278 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:12.315278 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:12.315278 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:12.315278 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:12.315278 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:12.316342 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:12.316342 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:12.316342 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:12.316342 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:12.316342 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:12.316342 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:12.316342 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:12.316342 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:12.316342 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:12.316342 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:12.316342 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:12.316342 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:12.316342 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:12.316342 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:12.316342 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:12.316342 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:12.316342 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:12.317378 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:12.317378 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:12.317378 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:12.317378 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 264/518] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:12.317378 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb050e56-82af-46d0-9910-3e734b17d661 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:12.338362 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4012aaa6-af83-4b06-9a2e-31a7ba882d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:12.339000 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4012aaa6-af83-4b06-9a2e-31a7ba882d3b None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:12.339275 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4012aaa6-af83-4b06-9a2e-31a7ba882d3b None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:12.339541 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4012aaa6-af83-4b06-9a2e-31a7ba882d3b None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:12.340170 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4012aaa6-af83-4b06-9a2e-31a7ba882d3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:12.340463 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4012aaa6-af83-4b06-9a2e-31a7ba882d3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:12.341803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4012aaa6-af83-4b06-9a2e-31a7ba882d3b None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:12.352884 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 255/519] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:12.354334 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4012aaa6-af83-4b06-9a2e-31a7ba882d3b None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:12.399091 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:12.408802 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:12.419268 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb050e56-82af-46d0-9910-3e734b17d661 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:12.451956 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384ebdb6-a0a4-4d03-925d-05088079ab3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:12.452555 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ebdb6-a0a4-4d03-925d-05088079ab3b None admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:12.452834 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ebdb6-a0a4-4d03-925d-05088079ab3b None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:12.453104 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ebdb6-a0a4-4d03-925d-05088079ab3b None admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:12.453741 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384ebdb6-a0a4-4d03-925d-05088079ab3b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:12.454113 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4012aaa6-af83-4b06-9a2e-31a7ba882d3b None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:12.455215 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384ebdb6-a0a4-4d03-925d-05088079ab3b None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:12.475530 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 265/520] 10.222.0.52 () {60 vars in 1132 bytes} [Sat Sep 24 14:13:12 2022] POST /identity/v3/users => generated 247 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:13:12.476511 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384ebdb6-a0a4-4d03-925d-05088079ab3b None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:12.488177 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fac104-d2ad-41b0-94ca-e613d04fb173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:12.488808 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fac104-d2ad-41b0-94ca-e613d04fb173 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:12.489087 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fac104-d2ad-41b0-94ca-e613d04fb173 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:12.489358 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fac104-d2ad-41b0-94ca-e613d04fb173 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:12.573629 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fac104-d2ad-41b0-94ca-e613d04fb173 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:12.575082 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fac104-d2ad-41b0-94ca-e613d04fb173 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:12.603111 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384ebdb6-a0a4-4d03-925d-05088079ab3b None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:12.637392 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:12.639649 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f74557-a3be-4989-bb70-60f30ca2e019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:12.640297 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f74557-a3be-4989-bb70-60f30ca2e019 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:12.640632 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f74557-a3be-4989-bb70-60f30ca2e019 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:12.640925 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f74557-a3be-4989-bb70-60f30ca2e019 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:12.645071 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:12.649883 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f74557-a3be-4989-bb70-60f30ca2e019 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:12.651407 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f74557-a3be-4989-bb70-60f30ca2e019 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:12.665951 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-64f74557-a3be-4989-bb70-60f30ca2e019 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:12.665951 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:12.665951 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:12.665951 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:12.665951 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:12.665951 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:12.665951 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:12.665951 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:12.665951 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:12.665951 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:12.665951 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:12.665951 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:12.665951 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:13:12.665951 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:13:12.665951 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:13:12.666799 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:13:12.666799 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:12.666799 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:12.666799 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:12.666799 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:12.666799 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:12.666799 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:12.666799 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:12.666799 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:12.666799 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:12.666799 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:12.666799 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:12.666799 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:12.666799 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:12.666799 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:12.666799 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:12.666799 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:12.667660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:13:12.667660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:13:12.667660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:13:12.667660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:13:12.667660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:13:12.667660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:13:12.667660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:12.667660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:12.671496 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 266/521] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:13:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:12.671745 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f74557-a3be-4989-bb70-60f30ca2e019 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:12.699802 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 256/522] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:13:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:13:12.700672 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fac104-d2ad-41b0-94ca-e613d04fb173 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:12.787441 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f74557-a3be-4989-bb70-60f30ca2e019 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:12.805676 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fac104-d2ad-41b0-94ca-e613d04fb173 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:12.826889 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f0822e-c032-45cd-9472-08ff8d8ed695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:12.827879 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f0822e-c032-45cd-9472-08ff8d8ed695 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:12.828492 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f0822e-c032-45cd-9472-08ff8d8ed695 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:12.828934 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f0822e-c032-45cd-9472-08ff8d8ed695 None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:12.829745 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f0822e-c032-45cd-9472-08ff8d8ed695 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:12.830331 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f0822e-c032-45cd-9472-08ff8d8ed695 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:12.831769 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f0822e-c032-45cd-9472-08ff8d8ed695 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:12.848726 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 267/523] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:12.848726 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f0822e-c032-45cd-9472-08ff8d8ed695 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:12.851554 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63734ae-b187-447c-a87f-3b22f16b2c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:12.852195 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63734ae-b187-447c-a87f-3b22f16b2c9e None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:12.852473 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63734ae-b187-447c-a87f-3b22f16b2c9e None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:12.852744 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63734ae-b187-447c-a87f-3b22f16b2c9e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:12.860417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63734ae-b187-447c-a87f-3b22f16b2c9e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:12.861935 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63734ae-b187-447c-a87f-3b22f16b2c9e None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:12.874257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-a63734ae-b187-447c-a87f-3b22f16b2c9e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 24 14:13:12.874257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:12.874257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:12.874257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:12.874257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:12.874257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:12.874257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:12.874257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:12.874257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:12.874257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:12.874257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:12.874257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:12.874257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:13:12.874257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:13:12.874257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:13:12.874257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:13:12.875206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:12.875206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:12.875206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:13:12.875206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:12.875206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:13:12.875206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:12.875206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:12.875206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:12.875206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:12.875206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:12.875206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:12.875206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:12.875206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:12.875206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:12.875206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:12.875206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:12.875206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:12.875206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:12.876078 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:12.876078 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:12.876078 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:13:12.876078 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:13:12.876078 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:13:12.876078 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:13:12.876078 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:13:12.876078 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:13:12.876078 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 24 14:13:12.876078 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:12.876078 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 257/524] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 14:13:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:12.876613 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63734ae-b187-447c-a87f-3b22f16b2c9e None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:12.892686 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:12.909326 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:12.959790 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f0822e-c032-45cd-9472-08ff8d8ed695 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:12.989237 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63734ae-b187-447c-a87f-3b22f16b2c9e None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:12.997378 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872560a9-393c-4b75-95ce-4332977a9e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:12.997986 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872560a9-393c-4b75-95ce-4332977a9e20 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:12.998268 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872560a9-393c-4b75-95ce-4332977a9e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:12.998538 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872560a9-393c-4b75-95ce-4332977a9e20 None admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:12.999272 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872560a9-393c-4b75-95ce-4332977a9e20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:12.999579 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872560a9-393c-4b75-95ce-4332977a9e20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:13.001041 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872560a9-393c-4b75-95ce-4332977a9e20 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:13.019968 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 268/525] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 14:13:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:13.020796 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872560a9-393c-4b75-95ce-4332977a9e20 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:13.032953 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a7c8ea-7a1a-467a-958e-2327f7b7dc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:13.033590 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a7c8ea-7a1a-467a-958e-2327f7b7dc00 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:13.033871 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a7c8ea-7a1a-467a-958e-2327f7b7dc00 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:13.034140 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a7c8ea-7a1a-467a-958e-2327f7b7dc00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:13.045424 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a7c8ea-7a1a-467a-958e-2327f7b7dc00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:13.046916 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a7c8ea-7a1a-467a-958e-2327f7b7dc00 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:13.062537 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-e6a7c8ea-7a1a-467a-958e-2327f7b7dc00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:13.062537 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:13.062537 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:13.062537 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:13.062537 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:13.062537 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:13.062537 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:13.062537 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:13.062537 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:13.062537 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:13.062537 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:13.062537 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:13.062537 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:13.062537 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:13.062537 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:13.062537 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:13.063511 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:13.063511 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:13.063511 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:13.063511 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:13.063511 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:13.063511 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:13.063511 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:13.063511 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:13.063511 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:13.063511 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:13.063511 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:13.063511 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:13.063511 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:13.063511 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:13.063511 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:13.063511 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:13.063511 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:13.064324 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:13.064324 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:13.064324 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:13.064324 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 258/526] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:13.065017 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a7c8ea-7a1a-467a-958e-2327f7b7dc00 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:13.133266 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872560a9-393c-4b75-95ce-4332977a9e20 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:13.141023 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:13.153474 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:13.173042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a3bf5-149d-4395-bea7-05bbb5df2924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:13.173675 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a3bf5-149d-4395-bea7-05bbb5df2924 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:13.173956 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a3bf5-149d-4395-bea7-05bbb5df2924 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:13.174231 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a3bf5-149d-4395-bea7-05bbb5df2924 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:13.178313 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a7c8ea-7a1a-467a-958e-2327f7b7dc00 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:13.181249 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5a3bf5-149d-4395-bea7-05bbb5df2924 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:13.182478 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5a3bf5-149d-4395-bea7-05bbb5df2924 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:13.193779 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-9d5a3bf5-149d-4395-bea7-05bbb5df2924 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:13.193779 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:13.193779 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:13.193779 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:13.193779 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:13.193779 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:13.193779 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:13.193779 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:13.193779 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:13.193779 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:13.193779 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:13.193779 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:13.193779 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:13:13.193779 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:13:13.193779 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:13:13.194653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:13:13.194653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:13.194653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:13.194653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:13.194653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:13.194653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:13.194653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:13.194653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:13.194653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:13.194653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:13.194653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:13.194653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:13.194653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:13.194653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:13.194653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:13.194653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:13.194653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:13.200820 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:13:13.200820 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:13:13.200820 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:13:13.200820 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:13:13.200820 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:13:13.200820 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:13:13.200820 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:13.200820 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:13.200820 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 269/527] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:13:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:13.200820 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a3bf5-149d-4395-bea7-05bbb5df2924 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:13.221202 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c130b21b-1f05-42e9-8f4e-7c4f627d395d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:13.221824 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130b21b-1f05-42e9-8f4e-7c4f627d395d None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:13.222100 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130b21b-1f05-42e9-8f4e-7c4f627d395d None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:13.222370 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130b21b-1f05-42e9-8f4e-7c4f627d395d None admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:13.223595 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c130b21b-1f05-42e9-8f4e-7c4f627d395d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:13.223909 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c130b21b-1f05-42e9-8f4e-7c4f627d395d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:13.225778 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c130b21b-1f05-42e9-8f4e-7c4f627d395d None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:13.270652 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 259/528] 10.222.0.52 () {56 vars in 1139 bytes} [Sat Sep 24 14:13:13 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:13.270652 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c130b21b-1f05-42e9-8f4e-7c4f627d395d None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:13.309538 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a3bf5-149d-4395-bea7-05bbb5df2924 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:13.343841 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a38b3a6-b068-43ce-bca3-c8ae1f37fb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:13.344443 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38b3a6-b068-43ce-bca3-c8ae1f37fb07 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:13.344714 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38b3a6-b068-43ce-bca3-c8ae1f37fb07 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:13.344978 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38b3a6-b068-43ce-bca3-c8ae1f37fb07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:13.345693 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a38b3a6-b068-43ce-bca3-c8ae1f37fb07 None admin] RBAC: Adding query filter params (role.id=be958e9954f04268a9f012c775fc17e5, scope.project.id=36a3ae7bab25471fb989136150b6438c, user.id=e82b01dbd6e84a9d82baa2c28d77fc28) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:13.345993 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a38b3a6-b068-43ce-bca3-c8ae1f37fb07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:13.347453 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a38b3a6-b068-43ce-bca3-c8ae1f37fb07 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:13.362849 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 270/529] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:13:13 2022] GET /identity/v3/role_assignments?user.id=e82b01dbd6e84a9d82baa2c28d77fc28&scope.project.id=36a3ae7bab25471fb989136150b6438c&role.id=be958e9954f04268a9f012c775fc17e5 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:13.362849 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a38b3a6-b068-43ce-bca3-c8ae1f37fb07 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:13.393563 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c130b21b-1f05-42e9-8f4e-7c4f627d395d None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:13.406589 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:13.440197 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:13.442416 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c055ddb-5768-47eb-a571-5c9b5a0116d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:13.443166 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c055ddb-5768-47eb-a571-5c9b5a0116d9 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:13.443576 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c055ddb-5768-47eb-a571-5c9b5a0116d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:13.443854 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c055ddb-5768-47eb-a571-5c9b5a0116d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:13.444487 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c055ddb-5768-47eb-a571-5c9b5a0116d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:13.444783 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c055ddb-5768-47eb-a571-5c9b5a0116d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:13.446179 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c055ddb-5768-47eb-a571-5c9b5a0116d9 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:13.464735 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 260/530] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:13.465714 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c055ddb-5768-47eb-a571-5c9b5a0116d9 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:13.489398 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a38b3a6-b068-43ce-bca3-c8ae1f37fb07 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:13.537593 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6ee8f-3b3f-4843-bd65-3203195fc863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:13.538220 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6ee8f-3b3f-4843-bd65-3203195fc863 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:13.538493 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6ee8f-3b3f-4843-bd65-3203195fc863 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:13.538757 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6ee8f-3b3f-4843-bd65-3203195fc863 None admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:13.539513 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf6ee8f-3b3f-4843-bd65-3203195fc863 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:13.539823 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf6ee8f-3b3f-4843-bd65-3203195fc863 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:13.541849 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf6ee8f-3b3f-4843-bd65-3203195fc863 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:13.554971 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 271/531] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 14:13:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:13.555817 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6ee8f-3b3f-4843-bd65-3203195fc863 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:13.613336 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c055ddb-5768-47eb-a571-5c9b5a0116d9 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:13.656875 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07edfb-2aee-4663-a023-80cb01e7e45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:13.658076 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07edfb-2aee-4663-a023-80cb01e7e45c None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:13.658969 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07edfb-2aee-4663-a023-80cb01e7e45c None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:13.659579 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07edfb-2aee-4663-a023-80cb01e7e45c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:13.669525 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:13.673439 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6ee8f-3b3f-4843-bd65-3203195fc863 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:13.717473 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b32cd43-4402-45e1-9cba-b82df6f95954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:13.718260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32cd43-4402-45e1-9cba-b82df6f95954 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:13.718550 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32cd43-4402-45e1-9cba-b82df6f95954 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:13.718845 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32cd43-4402-45e1-9cba-b82df6f95954 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:13.726858 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:13.727093 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b32cd43-4402-45e1-9cba-b82df6f95954 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:13.729179 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b32cd43-4402-45e1-9cba-b82df6f95954 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:13.743982 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-8b32cd43-4402-45e1-9cba-b82df6f95954 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:13.743982 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:13.743982 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:13.743982 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:13.743982 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:13.743982 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:13.743982 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:13.743982 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:13.743982 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:13.743982 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:13.743982 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:13.743982 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:13.743982 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:13:13.743982 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:13:13.743982 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:13:13.744838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:13:13.744838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:13.744838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:13.744838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:13.744838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:13.744838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:13.744838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:13.744838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:13.744838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:13.744838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:13.744838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:13.744838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:13.744838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:13.744838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:13.744838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:13.744838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:13.744838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:13.745662 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:13:13.745662 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:13:13.745662 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:13:13.745662 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:13:13.745662 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:13:13.745662 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:13:13.745662 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:13.745662 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:13.746714 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 272/532] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:13:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:13.746714 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b32cd43-4402-45e1-9cba-b82df6f95954 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:13.749208 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be07edfb-2aee-4663-a023-80cb01e7e45c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:13.752386 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be07edfb-2aee-4663-a023-80cb01e7e45c None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:13.871623 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b32cd43-4402-45e1-9cba-b82df6f95954 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:13.895771 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 261/533] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:13:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:13:13.896985 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07edfb-2aee-4663-a023-80cb01e7e45c None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:13.918534 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3832b-028c-439c-9dc6-eb48903a844d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:13.919363 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3832b-028c-439c-9dc6-eb48903a844d None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:13.919645 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3832b-028c-439c-9dc6-eb48903a844d None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:13.919917 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3832b-028c-439c-9dc6-eb48903a844d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:13.921038 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd3832b-028c-439c-9dc6-eb48903a844d None admin] RBAC: Adding query filter params (role.id=be958e9954f04268a9f012c775fc17e5, scope.project.id=36a3ae7bab25471fb989136150b6438c, user.id=79eaaae97aac4282a613939e5b768376) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:13.921352 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd3832b-028c-439c-9dc6-eb48903a844d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:13.922808 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd3832b-028c-439c-9dc6-eb48903a844d None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:13.930322 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 273/534] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:13:13 2022] GET /identity/v3/role_assignments?user.id=79eaaae97aac4282a613939e5b768376&scope.project.id=36a3ae7bab25471fb989136150b6438c&role.id=be958e9954f04268a9f012c775fc17e5 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:13.931641 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3832b-028c-439c-9dc6-eb48903a844d None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:13.970236 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:14.003473 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:14.024595 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07edfb-2aee-4663-a023-80cb01e7e45c None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:14.056997 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3832b-028c-439c-9dc6-eb48903a844d None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:14.070232 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde774bd-002b-49ee-bf19-3131cf9fd1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:14.071286 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde774bd-002b-49ee-bf19-3131cf9fd1f1 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:14.074652 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde774bd-002b-49ee-bf19-3131cf9fd1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:14.076618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde774bd-002b-49ee-bf19-3131cf9fd1f1 None admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:14.078299 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde774bd-002b-49ee-bf19-3131cf9fd1f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:14.079022 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde774bd-002b-49ee-bf19-3131cf9fd1f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:14.082157 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde774bd-002b-49ee-bf19-3131cf9fd1f1 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:14.121631 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 262/535] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 14:13:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:14.124769 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde774bd-002b-49ee-bf19-3131cf9fd1f1 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:14.131523 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2fd965-80df-4698-b5f6-1b2722ecfb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:14.132633 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2fd965-80df-4698-b5f6-1b2722ecfb7c None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:14.136081 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2fd965-80df-4698-b5f6-1b2722ecfb7c None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:14.136634 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2fd965-80df-4698-b5f6-1b2722ecfb7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:14.161539 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2fd965-80df-4698-b5f6-1b2722ecfb7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:14.164310 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2fd965-80df-4698-b5f6-1b2722ecfb7c None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:14.194415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-5c2fd965-80df-4698-b5f6-1b2722ecfb7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:14.194415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:14.194415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:14.194415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:14.194415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:14.194415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:14.194415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:14.194415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:14.194415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:14.194415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:14.194415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:14.194415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:14.194415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:14.194415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:14.194415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:14.194415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:14.195352 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:14.195352 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:14.195352 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:14.195352 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:14.195352 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:14.195352 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:14.195352 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:14.195352 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:14.195352 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:14.195352 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:14.195352 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:14.195352 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:14.195352 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:14.195352 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:14.195352 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:14.195352 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:14.195352 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:14.196179 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:14.196179 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:14.196179 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:14.199830 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 274/536] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:14.205272 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2fd965-80df-4698-b5f6-1b2722ecfb7c None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:14.320648 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:14.349940 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde774bd-002b-49ee-bf19-3131cf9fd1f1 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:14.363916 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2fd965-80df-4698-b5f6-1b2722ecfb7c None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:14.399370 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3321c111-9a81-4279-bcca-5bf46a43633d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:14.400513 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321c111-9a81-4279-bcca-5bf46a43633d None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:14.400804 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321c111-9a81-4279-bcca-5bf46a43633d None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:14.401075 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321c111-9a81-4279-bcca-5bf46a43633d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:14.401822 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3321c111-9a81-4279-bcca-5bf46a43633d None admin] RBAC: Adding query filter params (role.id=be958e9954f04268a9f012c775fc17e5, scope.project.id=36a3ae7bab25471fb989136150b6438c, user.id=63921623056045a9b97b2de3ca418138) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:14.404611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3321c111-9a81-4279-bcca-5bf46a43633d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:14.406169 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3321c111-9a81-4279-bcca-5bf46a43633d None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:14.420342 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 263/537] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:13:14 2022] GET /identity/v3/role_assignments?user.id=63921623056045a9b97b2de3ca418138&scope.project.id=36a3ae7bab25471fb989136150b6438c&role.id=be958e9954f04268a9f012c775fc17e5 => generated 259 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:14.441359 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c6bd4a-ec59-4972-9c31-eec2f12b2037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:14.442000 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6bd4a-ec59-4972-9c31-eec2f12b2037 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:14.442278 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6bd4a-ec59-4972-9c31-eec2f12b2037 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:14.442549 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6bd4a-ec59-4972-9c31-eec2f12b2037 None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:14.443244 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c6bd4a-ec59-4972-9c31-eec2f12b2037 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:14.443546 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c6bd4a-ec59-4972-9c31-eec2f12b2037 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:14.445031 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c6bd4a-ec59-4972-9c31-eec2f12b2037 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:14.465776 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 275/538] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:13:14.470969 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3321c111-9a81-4279-bcca-5bf46a43633d None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:14.495046 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:14.602482 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3321c111-9a81-4279-bcca-5bf46a43633d None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:14.638466 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c596b0-6e24-4fe2-9f4b-aec1724f889d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:14.639576 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c596b0-6e24-4fe2-9f4b-aec1724f889d None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:14.640047 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c596b0-6e24-4fe2-9f4b-aec1724f889d None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:14.640503 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c596b0-6e24-4fe2-9f4b-aec1724f889d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:14.647962 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c596b0-6e24-4fe2-9f4b-aec1724f889d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:14.649655 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c596b0-6e24-4fe2-9f4b-aec1724f889d None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:14.662678 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-31c596b0-6e24-4fe2-9f4b-aec1724f889d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:14.662678 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:14.662678 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:14.662678 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:14.662678 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:14.662678 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:14.662678 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:14.662678 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:14.662678 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:14.662678 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:14.662678 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:14.662678 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:14.662678 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:13:14.662678 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:13:14.662678 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:13:14.663556 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:13:14.663556 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:14.663556 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:14.663556 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:14.663556 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:14.663556 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:14.663556 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:14.663556 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:14.663556 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:14.663556 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:14.663556 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:14.663556 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:14.663556 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:14.663556 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:14.663556 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:14.663556 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:14.663556 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:14.664367 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:13:14.664367 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:13:14.664367 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:13:14.664367 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:13:14.664367 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:13:14.664367 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:13:14.664367 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:14.664367 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:14.666768 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 264/539] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:13:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:14.672023 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c6bd4a-ec59-4972-9c31-eec2f12b2037 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:14.796624 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c6bd4a-ec59-4972-9c31-eec2f12b2037 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:14.846567 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c0de9-c286-4d8e-8cbf-1949a441c8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:14.847210 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c0de9-c286-4d8e-8cbf-1949a441c8fe None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:14.847481 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c0de9-c286-4d8e-8cbf-1949a441c8fe None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:14.847753 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c0de9-c286-4d8e-8cbf-1949a441c8fe None admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:14.848437 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c0de9-c286-4d8e-8cbf-1949a441c8fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:14.848740 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c0de9-c286-4d8e-8cbf-1949a441c8fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:14.850173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c0de9-c286-4d8e-8cbf-1949a441c8fe None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:14.855978 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:14.865603 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 276/540] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 14:13:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:13:14.871611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c596b0-6e24-4fe2-9f4b-aec1724f889d None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:14.923033 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cec966-0071-46e6-bc00-6e7baf4188b2 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:14.923409 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cec966-0071-46e6-bc00-6e7baf4188b2 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:14.923790 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cec966-0071-46e6-bc00-6e7baf4188b2 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:14.925211 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 277/541] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:13:14.930317 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791a3c3-975f-439e-87f5-da40a0713660 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:14.930714 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791a3c3-975f-439e-87f5-da40a0713660 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:14.931116 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791a3c3-975f-439e-87f5-da40a0713660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:14.961303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-f791a3c3-975f-439e-87f5-da40a0713660 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:14.963969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-f791a3c3-975f-439e-87f5-da40a0713660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:14.981097 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:15.037152 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c596b0-6e24-4fe2-9f4b-aec1724f889d None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:15.097525 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d076c-2af1-4d52-a740-d5c9f8ce4681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:15.097778 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 278/542] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:13:15.098886 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d076c-2af1-4d52-a740-d5c9f8ce4681 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:15.099307 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d076c-2af1-4d52-a740-d5c9f8ce4681 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:15.099728 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d076c-2af1-4d52-a740-d5c9f8ce4681 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:15.100594 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936d076c-2af1-4d52-a740-d5c9f8ce4681 None admin] RBAC: Adding query filter params (role.id=be958e9954f04268a9f012c775fc17e5, scope.project.id=36a3ae7bab25471fb989136150b6438c, user.id=a9348164132846d2b593ddfa29a820eb) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:15.101037 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936d076c-2af1-4d52-a740-d5c9f8ce4681 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:15.102603 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936d076c-2af1-4d52-a740-d5c9f8ce4681 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:15.108963 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 265/543] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:13:14 2022] GET /identity/v3/role_assignments?user.id=a9348164132846d2b593ddfa29a820eb&scope.project.id=36a3ae7bab25471fb989136150b6438c&role.id=be958e9954f04268a9f012c775fc17e5 => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:15.422018 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791a3c3-975f-439e-87f5-da40a0713660 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:15.565475 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791a3c3-975f-439e-87f5-da40a0713660 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:15.601884 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:15.612660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca57d75d-5a6c-483e-b2d9-5fe1401731bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:15.613540 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca57d75d-5a6c-483e-b2d9-5fe1401731bf None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:15.613958 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca57d75d-5a6c-483e-b2d9-5fe1401731bf None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:15.614397 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca57d75d-5a6c-483e-b2d9-5fe1401731bf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:15.629952 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca57d75d-5a6c-483e-b2d9-5fe1401731bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:13:15.635375 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-ca57d75d-5a6c-483e-b2d9-5fe1401731bf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:13:15.635375 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:15.635375 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:15.635375 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:15.635375 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:15.635375 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:15.635375 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:15.635375 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:15.635375 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:15.635375 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:15.635375 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:15.635375 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:15.635375 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:13:15.635375 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:13:15.635375 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:13:15.635375 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:13:15.636426 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:13:15.636426 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:13:15.636426 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:13:15.636426 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:13:15.636426 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:15.636426 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:15.636426 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:15.636426 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:15.636426 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:15.636426 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:15.636426 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:15.636426 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:15.636426 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:15.636426 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:15.636426 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:15.636426 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:15.637384 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:15.637384 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:15.637384 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:15.637384 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:15.637384 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:13:15.637384 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:13:15.637384 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:13:15.637384 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:13:15.637384 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:13:15.637384 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:13:15.637384 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:13:15.637384 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:15.639374 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 279/544] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 14:13:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:15.645024 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d076c-2af1-4d52-a740-d5c9f8ce4681 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:15.762836 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d076c-2af1-4d52-a740-d5c9f8ce4681 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:15.792326 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:15.797523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c360a6c1-e2c4-4a2e-a882-dd089c1e2953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:15.798127 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360a6c1-e2c4-4a2e-a882-dd089c1e2953 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:15.798406 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360a6c1-e2c4-4a2e-a882-dd089c1e2953 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:15.798745 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360a6c1-e2c4-4a2e-a882-dd089c1e2953 None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:15.799439 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c360a6c1-e2c4-4a2e-a882-dd089c1e2953 None admin] RBAC: Adding query filter params (name=service) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:15.799828 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c360a6c1-e2c4-4a2e-a882-dd089c1e2953 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:15.801304 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c360a6c1-e2c4-4a2e-a882-dd089c1e2953 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:15.813747 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 266/545] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 14:13:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:15.819558 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca57d75d-5a6c-483e-b2d9-5fe1401731bf None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:15.885681 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60748dc-2244-4588-be6c-78cb4c1fc573 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:15.885962 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60748dc-2244-4588-be6c-78cb4c1fc573 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:15.886231 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60748dc-2244-4588-be6c-78cb4c1fc573 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:15.887852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 267/546] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:15.892884 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a16950-7597-4ad5-95df-9bf9e006b85f None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:15.893169 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a16950-7597-4ad5-95df-9bf9e006b85f None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:15.893446 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a16950-7597-4ad5-95df-9bf9e006b85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:15.929576 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-97a16950-7597-4ad5-95df-9bf9e006b85f None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:15.933970 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-97a16950-7597-4ad5-95df-9bf9e006b85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:16.002779 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca57d75d-5a6c-483e-b2d9-5fe1401731bf None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:16.014321 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 268/547] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:13:16.068752 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e170b991-902d-42f6-9767-89649bcd9c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:16.069396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170b991-902d-42f6-9767-89649bcd9c30 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:16.069671 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170b991-902d-42f6-9767-89649bcd9c30 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:16.069940 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170b991-902d-42f6-9767-89649bcd9c30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:16.085703 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e170b991-902d-42f6-9767-89649bcd9c30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:16.087208 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e170b991-902d-42f6-9767-89649bcd9c30 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:16.105639 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-e170b991-902d-42f6-9767-89649bcd9c30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:16.105639 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:16.105639 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:16.105639 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:16.105639 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:16.105639 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:16.105639 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:16.105639 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:16.105639 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:16.105639 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:16.105639 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:16.105639 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:16.105639 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:16.105639 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:16.105639 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:16.105639 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:16.106981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:16.106981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:16.106981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:16.106981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:16.106981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:16.106981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:16.106981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:16.106981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:16.106981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:16.106981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:16.106981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:16.106981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:16.106981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:16.106981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:16.106981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:16.106981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:16.106981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:16.107831 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:16.107831 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:16.107831 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:16.107831 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 280/548] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:16.112727 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a16950-7597-4ad5-95df-9bf9e006b85f None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:16.238874 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a16950-7597-4ad5-95df-9bf9e006b85f None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:16.266449 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:16.280179 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152b7041-5744-4902-89bd-a7bb97678573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:16.280786 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b7041-5744-4902-89bd-a7bb97678573 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:16.281063 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b7041-5744-4902-89bd-a7bb97678573 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:16.281332 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b7041-5744-4902-89bd-a7bb97678573 None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:16.281968 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152b7041-5744-4902-89bd-a7bb97678573 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:16.282264 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152b7041-5744-4902-89bd-a7bb97678573 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:16.283926 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152b7041-5744-4902-89bd-a7bb97678573 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:16.298505 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 269/549] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:16.303912 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e170b991-902d-42f6-9767-89649bcd9c30 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:16.360695 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:16.393615 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152b7041-5744-4902-89bd-a7bb97678573 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:16.442537 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e170b991-902d-42f6-9767-89649bcd9c30 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:16.515555 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3138b3-0ca2-4f07-b4cc-f3dda7cd90f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:16.516251 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3138b3-0ca2-4f07-b4cc-f3dda7cd90f5 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:16.516585 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3138b3-0ca2-4f07-b4cc-f3dda7cd90f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:16.516806 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3138b3-0ca2-4f07-b4cc-f3dda7cd90f5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:16.535908 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3138b3-0ca2-4f07-b4cc-f3dda7cd90f5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:16.537375 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3138b3-0ca2-4f07-b4cc-f3dda7cd90f5 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:16.538321 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152b7041-5744-4902-89bd-a7bb97678573 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:16.564018 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-db3138b3-0ca2-4f07-b4cc-f3dda7cd90f5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 24 14:13:16.564018 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:16.564018 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:16.564018 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:16.564018 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:16.564018 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:16.564018 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:16.564018 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:16.564018 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:16.564018 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:16.564018 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:16.564018 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:16.564018 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:13:16.564018 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:13:16.564018 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:13:16.564018 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:13:16.565199 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:16.565199 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:16.565199 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:13:16.565199 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:16.565199 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:13:16.565199 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:16.565199 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:16.565199 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:16.565199 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:16.565199 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:16.565199 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:16.565199 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:16.565199 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:16.565199 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:16.565199 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:16.565199 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:16.565199 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:16.565199 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:16.566091 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:16.566091 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:16.566091 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:13:16.566091 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:13:16.566091 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:13:16.566091 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:13:16.566091 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:13:16.566091 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:13:16.566091 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 24 14:13:16.566091 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:16.566091 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 281/550] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 14:13:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:16.575911 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3138b3-0ca2-4f07-b4cc-f3dda7cd90f5 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:16.579059 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf7a286-c94e-4eb3-9e56-f1dc5c66a884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:16.579714 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7a286-c94e-4eb3-9e56-f1dc5c66a884 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:16.579991 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7a286-c94e-4eb3-9e56-f1dc5c66a884 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:16.580262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7a286-c94e-4eb3-9e56-f1dc5c66a884 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:16.593118 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf7a286-c94e-4eb3-9e56-f1dc5c66a884 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:13:16.598632 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-0cf7a286-c94e-4eb3-9e56-f1dc5c66a884 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:13:16.598632 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:16.598632 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:16.598632 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:16.598632 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:16.598632 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:16.598632 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:16.598632 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:16.598632 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:16.598632 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:16.598632 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:16.598632 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:16.598632 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:13:16.598632 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:13:16.598632 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:13:16.598632 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:13:16.599567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:13:16.599567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:13:16.599567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:13:16.599567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:13:16.599567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:16.599567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:16.599567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:16.599567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:16.599567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:16.599567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:16.599567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:16.599567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:16.599567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:16.599567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:16.599567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:16.599567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:16.601440 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:16.601440 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:16.601440 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:16.601440 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:16.601440 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:13:16.601440 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:13:16.601440 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:13:16.601440 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:13:16.601440 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:13:16.601440 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:13:16.601440 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:13:16.601440 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:16.601440 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 270/551] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 14:13:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:13:16.606602 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf7a286-c94e-4eb3-9e56-f1dc5c66a884 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:16.671129 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:16.742631 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf7a286-c94e-4eb3-9e56-f1dc5c66a884 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:16.771278 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:16.775337 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3138b3-0ca2-4f07-b4cc-f3dda7cd90f5 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:16.788492 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58351d92-d78c-4166-b54e-d6fb38ecd8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:16.789111 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58351d92-d78c-4166-b54e-d6fb38ecd8a6 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:16.789390 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58351d92-d78c-4166-b54e-d6fb38ecd8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:16.789659 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58351d92-d78c-4166-b54e-d6fb38ecd8a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:16.790581 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58351d92-d78c-4166-b54e-d6fb38ecd8a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:16.791455 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58351d92-d78c-4166-b54e-d6fb38ecd8a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:16.792840 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58351d92-d78c-4166-b54e-d6fb38ecd8a6 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:16.804261 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 271/552] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 14:13:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:13:16.807718 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647598ae-60f1-4c8d-b16f-a5e79a139b72 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:16.808210 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647598ae-60f1-4c8d-b16f-a5e79a139b72 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:16.808677 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647598ae-60f1-4c8d-b16f-a5e79a139b72 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:16.811064 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 272/553] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:16.812279 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647598ae-60f1-4c8d-b16f-a5e79a139b72 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:16.829918 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8febbc48-bcd4-4484-b866-95f31cdd52a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:16.831073 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8febbc48-bcd4-4484-b866-95f31cdd52a5 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:16.831643 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8febbc48-bcd4-4484-b866-95f31cdd52a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:16.832109 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8febbc48-bcd4-4484-b866-95f31cdd52a5 None admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:16.833090 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8febbc48-bcd4-4484-b866-95f31cdd52a5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:16.833624 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8febbc48-bcd4-4484-b866-95f31cdd52a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:16.835457 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8febbc48-bcd4-4484-b866-95f31cdd52a5 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:16.858482 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 282/554] 10.222.0.52 () {56 vars in 1135 bytes} [Sat Sep 24 14:13:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:16.860139 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0fccd-bcc8-4f61-9f80-be7a8ee7a12e None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:16.860765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0fccd-bcc8-4f61-9f80-be7a8ee7a12e None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:16.861238 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0fccd-bcc8-4f61-9f80-be7a8ee7a12e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:16.880284 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-afb0fccd-bcc8-4f61-9f80-be7a8ee7a12e None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:16.882450 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-afb0fccd-bcc8-4f61-9f80-be7a8ee7a12e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:16.942831 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647598ae-60f1-4c8d-b16f-a5e79a139b72 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:16.953411 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:16.979425 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 283/555] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:13:16.980401 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb0fccd-bcc8-4f61-9f80-be7a8ee7a12e None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:17.007091 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2058e17-bcf8-447f-a6dd-137dc6d476a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:17.008677 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2058e17-bcf8-447f-a6dd-137dc6d476a4 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:17.009994 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2058e17-bcf8-447f-a6dd-137dc6d476a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:17.010417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2058e17-bcf8-447f-a6dd-137dc6d476a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:17.039868 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2058e17-bcf8-447f-a6dd-137dc6d476a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:17.041348 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2058e17-bcf8-447f-a6dd-137dc6d476a4 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:17.065460 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-b2058e17-bcf8-447f-a6dd-137dc6d476a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:17.065460 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:17.065460 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:17.065460 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:17.065460 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:17.065460 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:17.065460 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:17.065460 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:17.065460 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:17.065460 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:17.065460 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:17.065460 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:17.065460 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:17.065460 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:17.065460 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:17.065460 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:17.066406 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:17.066406 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:17.066406 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:17.066406 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:17.066406 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:17.066406 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:17.066406 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:17.066406 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:17.066406 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:17.066406 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:17.066406 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:17.066406 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:17.066406 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:17.066406 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:17.066406 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:17.066406 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:17.066406 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:17.067300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:17.067300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:17.067300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:17.073642 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 273/556] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:17.092914 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2058e17-bcf8-447f-a6dd-137dc6d476a4 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:17.117017 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:17.134005 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb0fccd-bcc8-4f61-9f80-be7a8ee7a12e None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:17.176053 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eeac9e-01e4-4949-b41d-8669168cfe97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:17.176674 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eeac9e-01e4-4949-b41d-8669168cfe97 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:17.176955 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eeac9e-01e4-4949-b41d-8669168cfe97 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:17.177288 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eeac9e-01e4-4949-b41d-8669168cfe97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:17.217484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2058e17-bcf8-447f-a6dd-137dc6d476a4 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:17.263060 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee98701b-32ae-42d5-bc4d-de1a60816c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:17.263835 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee98701b-32ae-42d5-bc4d-de1a60816c7f None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:17.264203 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee98701b-32ae-42d5-bc4d-de1a60816c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:17.264769 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee98701b-32ae-42d5-bc4d-de1a60816c7f None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:17.265896 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee98701b-32ae-42d5-bc4d-de1a60816c7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:17.266279 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee98701b-32ae-42d5-bc4d-de1a60816c7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:17.268043 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee98701b-32ae-42d5-bc4d-de1a60816c7f None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:17.272999 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eeac9e-01e4-4949-b41d-8669168cfe97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:17.274726 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eeac9e-01e4-4949-b41d-8669168cfe97 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:17.286927 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 274/557] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:17.288998 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafec09b-5d3c-41a9-9e69-25f54ab7b1ae None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:17.289293 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafec09b-5d3c-41a9-9e69-25f54ab7b1ae None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:17.289572 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafec09b-5d3c-41a9-9e69-25f54ab7b1ae None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:17.291975 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 275/558] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:17.292512 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafec09b-5d3c-41a9-9e69-25f54ab7b1ae None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:17.315468 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:17.365856 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:17.382491 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 284/559] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:13:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:13:17.383192 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eeac9e-01e4-4949-b41d-8669168cfe97 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:17.411935 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafec09b-5d3c-41a9-9e69-25f54ab7b1ae None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:17.452658 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de5b24b-569a-4830-ba89-f062c4d8611b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:17.453459 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5b24b-569a-4830-ba89-f062c4d8611b None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:17.453741 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5b24b-569a-4830-ba89-f062c4d8611b None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:17.454013 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5b24b-569a-4830-ba89-f062c4d8611b None admin] PATH_INFO: `/v3/users/glance` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:17.461680 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de5b24b-569a-4830-ba89-f062c4d8611b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:17.463754 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de5b24b-569a-4830-ba89-f062c4d8611b None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:17.475487 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-4de5b24b-569a-4830-ba89-f062c4d8611b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 24 14:13:17.475487 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:17.475487 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:17.475487 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:17.475487 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:17.475487 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:17.475487 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:17.475487 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:17.475487 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:17.475487 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:17.475487 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:17.475487 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:17.475487 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:13:17.475487 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:13:17.475487 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:13:17.475487 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:13:17.476420 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:17.476420 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:17.476420 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:13:17.476420 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:17.476420 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:13:17.476420 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:17.476420 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:17.476420 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:17.476420 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:17.476420 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:17.476420 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:17.476420 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:17.476420 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:17.476420 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:17.476420 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:17.476420 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:17.476420 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:17.476420 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:17.477334 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:17.477334 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:17.477334 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:13:17.477334 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:13:17.477334 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:13:17.477334 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:13:17.477334 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:13:17.477334 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:13:17.477334 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 24 14:13:17.477334 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:17.477334 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 276/560] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:13:17 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:17.477913 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64e9b8-2a9d-4fdd-9260-d5e15153110d None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:17.477982 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64e9b8-2a9d-4fdd-9260-d5e15153110d None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:17.479064 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64e9b8-2a9d-4fdd-9260-d5e15153110d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:17.490667 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eeac9e-01e4-4949-b41d-8669168cfe97 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:17.497355 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-ce64e9b8-2a9d-4fdd-9260-d5e15153110d None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:17.499004 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-ce64e9b8-2a9d-4fdd-9260-d5e15153110d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:17.529064 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27fb894-2ce2-4fd3-a79e-80b888fb79a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:17.529695 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27fb894-2ce2-4fd3-a79e-80b888fb79a4 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:17.529971 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27fb894-2ce2-4fd3-a79e-80b888fb79a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:17.530240 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27fb894-2ce2-4fd3-a79e-80b888fb79a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:17.542880 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27fb894-2ce2-4fd3-a79e-80b888fb79a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:13:17.548572 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-c27fb894-2ce2-4fd3-a79e-80b888fb79a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:13:17.548572 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:17.548572 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:17.548572 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:17.548572 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:17.548572 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:17.548572 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:17.548572 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:17.548572 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:17.548572 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:17.548572 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:17.548572 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:17.548572 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:13:17.548572 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:13:17.548572 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:13:17.548572 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:13:17.549629 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:13:17.549629 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:13:17.549629 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:13:17.549629 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:13:17.549629 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:17.549629 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:17.549629 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:17.549629 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:17.549629 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:17.549629 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:17.549629 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:17.549629 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:17.549629 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:17.549629 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:17.549629 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:17.549629 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:17.550466 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:17.550466 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:17.550466 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:17.550466 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:17.550466 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:13:17.550466 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:13:17.550466 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:13:17.550466 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:13:17.550466 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:13:17.550466 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:13:17.550466 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:13:17.550466 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:17.550466 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 285/561] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 14:13:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:17.550466 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27fb894-2ce2-4fd3-a79e-80b888fb79a4 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:17.575914 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 277/562] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:13:17.577294 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce64e9b8-2a9d-4fdd-9260-d5e15153110d None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:17.597717 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:17.610077 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:17.689622 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27fb894-2ce2-4fd3-a79e-80b888fb79a4 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:17.706304 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce64e9b8-2a9d-4fdd-9260-d5e15153110d None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:17.736849 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92032391-101c-4c61-8114-b69124f1e18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:17.737491 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92032391-101c-4c61-8114-b69124f1e18d None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:17.737769 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92032391-101c-4c61-8114-b69124f1e18d None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:17.738043 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92032391-101c-4c61-8114-b69124f1e18d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:17.741923 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c458d28-ca25-47ea-b86f-a9471212f870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:17.742529 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c458d28-ca25-47ea-b86f-a9471212f870 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:17.742806 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c458d28-ca25-47ea-b86f-a9471212f870 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:17.743104 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c458d28-ca25-47ea-b86f-a9471212f870 None admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:17.744018 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c458d28-ca25-47ea-b86f-a9471212f870 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:17.744226 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c458d28-ca25-47ea-b86f-a9471212f870 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:17.745640 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c458d28-ca25-47ea-b86f-a9471212f870 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:17.748422 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92032391-101c-4c61-8114-b69124f1e18d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:17.749863 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92032391-101c-4c61-8114-b69124f1e18d None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:17.767914 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 278/563] 10.222.0.52 () {56 vars in 1137 bytes} [Sat Sep 24 14:13:17 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:17.767914 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c458d28-ca25-47ea-b86f-a9471212f870 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:17.768182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-92032391-101c-4c61-8114-b69124f1e18d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:17.768182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:17.768182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:17.768182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:17.768182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:17.768182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:17.768182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:17.768182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:17.768182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:17.768182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:17.768182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:17.768182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:17.768182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:17.768182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:17.768182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:17.768182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:17.769008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:17.769008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:17.769008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:17.769008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:17.769008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:17.769008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:17.769008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:17.769008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:17.769008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:17.769008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:17.769008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:17.769008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:17.769008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:17.769008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:17.769008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:17.769008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:17.769008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:17.769876 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:17.769876 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:17.769876 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:17.770881 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 286/564] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:17.771903 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a7123-3a71-45d1-97f2-198838bed752 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:17.772181 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a7123-3a71-45d1-97f2-198838bed752 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:17.772448 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a7123-3a71-45d1-97f2-198838bed752 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:17.773690 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 287/565] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:17.774375 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a7123-3a71-45d1-97f2-198838bed752 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:17.850656 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:17.888953 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c458d28-ca25-47ea-b86f-a9471212f870 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:17.889278 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:17.894800 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a7123-3a71-45d1-97f2-198838bed752 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:17.933660 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53db88f-2dc2-4253-9c1f-10e8860ce4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:17.934276 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53db88f-2dc2-4253-9c1f-10e8860ce4cf None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:17.934563 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53db88f-2dc2-4253-9c1f-10e8860ce4cf None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:17.934897 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53db88f-2dc2-4253-9c1f-10e8860ce4cf None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:17.935601 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53db88f-2dc2-4253-9c1f-10e8860ce4cf None admin] RBAC: Adding query filter params (name=service) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:17.935720 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee5b225-cd6f-4272-b7de-5b5ed66b82e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:17.935904 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53db88f-2dc2-4253-9c1f-10e8860ce4cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:17.936321 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5b225-cd6f-4272-b7de-5b5ed66b82e5 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:17.936597 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5b225-cd6f-4272-b7de-5b5ed66b82e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:17.936865 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5b225-cd6f-4272-b7de-5b5ed66b82e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:17.937272 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53db88f-2dc2-4253-9c1f-10e8860ce4cf None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:17.946563 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 279/566] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 14:13:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:17.947985 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53db88f-2dc2-4253-9c1f-10e8860ce4cf None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:17.949135 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee5b225-cd6f-4272-b7de-5b5ed66b82e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:17.950801 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee5b225-cd6f-4272-b7de-5b5ed66b82e5 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:17.966636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-9ee5b225-cd6f-4272-b7de-5b5ed66b82e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:17.966636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:17.966636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:17.966636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:17.966636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:17.966636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:17.966636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:17.966636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:17.966636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:17.966636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:17.966636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:17.966636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:17.966636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:17.966636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:17.966636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:17.966636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:17.967769 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:17.967769 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:17.967769 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:17.967769 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:17.967769 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:17.967769 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:17.967769 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:17.967769 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:17.967769 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:17.967769 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:17.967769 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:17.967769 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:17.967769 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:17.967769 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:17.967769 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:17.967769 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:17.967769 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:17.968842 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:17.968842 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:17.968842 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:17.970554 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 288/567] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:17.971960 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e45f5f-9237-4299-9280-2cddc9ddcc9a None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:17.972384 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e45f5f-9237-4299-9280-2cddc9ddcc9a None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:17.972825 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e45f5f-9237-4299-9280-2cddc9ddcc9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:17.991858 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-c7e45f5f-9237-4299-9280-2cddc9ddcc9a None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:17.993290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-c7e45f5f-9237-4299-9280-2cddc9ddcc9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:18.061043 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 289/568] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:13:18.061394 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53db88f-2dc2-4253-9c1f-10e8860ce4cf None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:18.061877 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e45f5f-9237-4299-9280-2cddc9ddcc9a None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:18.093776 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:18.101892 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f6a6ce-163c-4e55-a010-a7285bcc1166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:18.102500 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6a6ce-163c-4e55-a010-a7285bcc1166 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:18.102780 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6a6ce-163c-4e55-a010-a7285bcc1166 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:18.103083 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6a6ce-163c-4e55-a010-a7285bcc1166 None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:18.103756 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f6a6ce-163c-4e55-a010-a7285bcc1166 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:18.104084 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f6a6ce-163c-4e55-a010-a7285bcc1166 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:18.105453 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f6a6ce-163c-4e55-a010-a7285bcc1166 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:18.123814 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 280/569] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:18.123814 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f6a6ce-163c-4e55-a010-a7285bcc1166 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:18.199770 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:18.220807 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e45f5f-9237-4299-9280-2cddc9ddcc9a None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:18.242330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f6a6ce-163c-4e55-a010-a7285bcc1166 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:18.256420 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f8f7a-9fc4-434f-ac8d-f3306ec6af41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:18.257158 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f8f7a-9fc4-434f-ac8d-f3306ec6af41 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:18.257437 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f8f7a-9fc4-434f-ac8d-f3306ec6af41 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:18.257708 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f8f7a-9fc4-434f-ac8d-f3306ec6af41 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:18.268534 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997f8f7a-9fc4-434f-ac8d-f3306ec6af41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:13:18.273248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-997f8f7a-9fc4-434f-ac8d-f3306ec6af41 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:13:18.273248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:18.273248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:18.273248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:18.273248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:18.273248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:18.273248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:18.273248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:18.273248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:18.273248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:18.273248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:18.273248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:18.273248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:13:18.273248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:13:18.273248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:13:18.273248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:13:18.274161 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:13:18.274161 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:13:18.274161 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:13:18.274161 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:13:18.274161 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:18.274161 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:18.274161 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:18.274161 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:18.274161 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:18.274161 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:18.274161 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:18.274161 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:18.274161 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:18.274161 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:18.274161 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:18.274161 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:18.275638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:18.275638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:18.275638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:18.275638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:18.275638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:13:18.275638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:13:18.275638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:13:18.275638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:13:18.275638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:13:18.275638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:13:18.275638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:13:18.275638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:18.275638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 290/570] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 14:13:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:18.276804 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f8f7a-9fc4-434f-ac8d-f3306ec6af41 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:18.284301 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2be414-b148-49f7-b1d8-cead3d16371b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:18.284931 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2be414-b148-49f7-b1d8-cead3d16371b None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:18.285216 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2be414-b148-49f7-b1d8-cead3d16371b None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:18.285500 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2be414-b148-49f7-b1d8-cead3d16371b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:18.295950 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2be414-b148-49f7-b1d8-cead3d16371b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:18.297810 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2be414-b148-49f7-b1d8-cead3d16371b None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:18.314262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-8a2be414-b148-49f7-b1d8-cead3d16371b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:18.314262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:18.314262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:18.314262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:18.314262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:18.314262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:18.314262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:18.314262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:18.314262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:18.314262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:18.314262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:18.314262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:18.314262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:18.314262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:18.314262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:18.314262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:18.315438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:18.315438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:18.315438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:18.315438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:18.315438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:18.315438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:18.315438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:18.315438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:18.315438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:18.315438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:18.315438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:18.315438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:18.315438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:18.315438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:18.315438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:18.315438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:18.315438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:18.316298 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:18.316298 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:18.316298 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:18.321097 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 281/571] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:18.322426 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2be414-b148-49f7-b1d8-cead3d16371b None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:18.364115 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:18.381183 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f8f7a-9fc4-434f-ac8d-f3306ec6af41 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:18.415977 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb35079-d76d-48b4-a221-e5dbca22ac59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:18.416820 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb35079-d76d-48b4-a221-e5dbca22ac59 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:18.417104 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb35079-d76d-48b4-a221-e5dbca22ac59 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:18.417374 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb35079-d76d-48b4-a221-e5dbca22ac59 None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:18.418010 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb35079-d76d-48b4-a221-e5dbca22ac59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:18.418308 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb35079-d76d-48b4-a221-e5dbca22ac59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:18.419712 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb35079-d76d-48b4-a221-e5dbca22ac59 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:18.422177 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2be414-b148-49f7-b1d8-cead3d16371b None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:18.427798 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:18.431618 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 291/572] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:18.432684 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb35079-d76d-48b4-a221-e5dbca22ac59 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:18.457222 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae089b74-2c08-498a-9f7a-f140c8bcc736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:18.457848 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae089b74-2c08-498a-9f7a-f140c8bcc736 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:18.458121 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae089b74-2c08-498a-9f7a-f140c8bcc736 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:18.458386 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae089b74-2c08-498a-9f7a-f140c8bcc736 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:18.465364 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae089b74-2c08-498a-9f7a-f140c8bcc736 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:18.466782 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae089b74-2c08-498a-9f7a-f140c8bcc736 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:18.477898 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-ae089b74-2c08-498a-9f7a-f140c8bcc736 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:18.477898 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:18.477898 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:18.477898 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:18.477898 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:18.477898 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:18.477898 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:18.477898 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:18.477898 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:18.477898 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:18.477898 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:18.477898 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:18.477898 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:13:18.477898 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:13:18.477898 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:13:18.479374 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:13:18.479374 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:18.479374 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:18.479374 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:18.479374 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:18.479374 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:18.479374 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:18.479374 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:18.479374 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:18.479374 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:18.479374 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:18.479374 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:18.479374 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:18.479374 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:18.479374 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:18.479374 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:18.479374 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:18.480250 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:13:18.480250 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:13:18.480250 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:13:18.480250 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:13:18.480250 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:13:18.480250 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:13:18.480250 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:18.480250 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:18.480250 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 282/573] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:13:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:18.480250 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae089b74-2c08-498a-9f7a-f140c8bcc736 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:18.534865 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb35079-d76d-48b4-a221-e5dbca22ac59 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:18.567414 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4e1c9-a94d-4200-abbe-c4aad6cd81a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:18.568010 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4e1c9-a94d-4200-abbe-c4aad6cd81a6 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:18.568290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4e1c9-a94d-4200-abbe-c4aad6cd81a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:18.568560 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4e1c9-a94d-4200-abbe-c4aad6cd81a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:18.569209 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd4e1c9-a94d-4200-abbe-c4aad6cd81a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:18.569510 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd4e1c9-a94d-4200-abbe-c4aad6cd81a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:18.570889 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd4e1c9-a94d-4200-abbe-c4aad6cd81a6 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:18.576775 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:18.579756 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae089b74-2c08-498a-9f7a-f140c8bcc736 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:18.580076 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 292/574] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 14:13:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:18.580866 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4e1c9-a94d-4200-abbe-c4aad6cd81a6 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:18.613194 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620d6615-e53f-42df-b604-2dfd61ae15b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:18.613802 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620d6615-e53f-42df-b604-2dfd61ae15b7 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:18.614083 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620d6615-e53f-42df-b604-2dfd61ae15b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:18.614403 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620d6615-e53f-42df-b604-2dfd61ae15b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:18.615031 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620d6615-e53f-42df-b604-2dfd61ae15b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:18.615334 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620d6615-e53f-42df-b604-2dfd61ae15b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:18.616694 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620d6615-e53f-42df-b604-2dfd61ae15b7 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:18.628778 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 283/575] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:18.629634 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620d6615-e53f-42df-b604-2dfd61ae15b7 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:18.645659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:18.683990 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4e1c9-a94d-4200-abbe-c4aad6cd81a6 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:18.716829 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de85ee8a-2be8-4639-8419-64fdcf74fdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:18.717442 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85ee8a-2be8-4639-8419-64fdcf74fdae None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:18.717719 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85ee8a-2be8-4639-8419-64fdcf74fdae None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:18.717985 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85ee8a-2be8-4639-8419-64fdcf74fdae None admin] PATH_INFO: `/v3/roles/service` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:18.727648 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620d6615-e53f-42df-b604-2dfd61ae15b7 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:18.727890 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de85ee8a-2be8-4639-8419-64fdcf74fdae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:13:18.734401 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-de85ee8a-2be8-4639-8419-64fdcf74fdae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:13:18.734401 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:18.734401 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:18.734401 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:18.734401 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:18.734401 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:18.734401 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:18.734401 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:18.734401 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:18.734401 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:18.734401 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:18.734401 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:18.734401 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:13:18.734401 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:13:18.734401 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:13:18.734401 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:13:18.735404 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:13:18.735404 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:13:18.735404 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:13:18.735404 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:13:18.735404 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:18.735404 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:18.735404 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:18.735404 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:18.735404 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:18.735404 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:18.735404 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:18.735404 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:18.735404 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:18.735404 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:18.735404 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:18.735404 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:18.736246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:18.736246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:18.736246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:18.736246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:18.736246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:13:18.736246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:13:18.736246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:13:18.736246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:13:18.736246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:13:18.736246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:13:18.736246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:13:18.736246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:18.736246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 293/576] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 14:13:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:18.736246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de85ee8a-2be8-4639-8419-64fdcf74fdae None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:18.761124 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdc2349-dc2c-42c4-bd84-41bc7fe29007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:18.761749 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc2349-dc2c-42c4-bd84-41bc7fe29007 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:18.762030 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc2349-dc2c-42c4-bd84-41bc7fe29007 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:18.762299 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc2349-dc2c-42c4-bd84-41bc7fe29007 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:18.769164 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdc2349-dc2c-42c4-bd84-41bc7fe29007 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:18.770588 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdc2349-dc2c-42c4-bd84-41bc7fe29007 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:18.782325 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-ffdc2349-dc2c-42c4-bd84-41bc7fe29007 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:18.782325 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:18.782325 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:18.782325 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:18.782325 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:18.782325 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:18.782325 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:18.782325 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:18.782325 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:18.782325 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:18.782325 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:18.782325 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:18.782325 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:13:18.782325 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:13:18.782325 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:13:18.783582 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:13:18.783582 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:18.783582 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:18.783582 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:18.783582 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:18.783582 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:18.783582 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:18.783582 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:18.783582 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:18.783582 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:18.783582 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:18.783582 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:18.783582 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:18.783582 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:18.783582 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:18.783582 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:18.783582 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:18.784651 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:13:18.784651 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:13:18.784651 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:13:18.784651 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:13:18.784651 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:13:18.784651 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:13:18.784651 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:18.784651 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:18.784651 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 284/577] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:13:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:18.785331 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdc2349-dc2c-42c4-bd84-41bc7fe29007 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:18.809504 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:18.849271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de85ee8a-2be8-4639-8419-64fdcf74fdae None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:18.871709 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:18.883936 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b056508e-a883-49e8-b359-af73fd782f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:18.884554 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056508e-a883-49e8-b359-af73fd782f07 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:18.885332 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056508e-a883-49e8-b359-af73fd782f07 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:18.885605 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056508e-a883-49e8-b359-af73fd782f07 None admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:18.886316 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b056508e-a883-49e8-b359-af73fd782f07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:18.886617 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b056508e-a883-49e8-b359-af73fd782f07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:18.888070 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdc2349-dc2c-42c4-bd84-41bc7fe29007 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:18.888185 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b056508e-a883-49e8-b359-af73fd782f07 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:18.904914 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 294/578] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 14:13:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:18.904914 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b056508e-a883-49e8-b359-af73fd782f07 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:18.922509 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4d8d0-3044-4515-825d-06002bc0222c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:18.923160 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4d8d0-3044-4515-825d-06002bc0222c None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:18.923441 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4d8d0-3044-4515-825d-06002bc0222c None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:18.923708 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4d8d0-3044-4515-825d-06002bc0222c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:18.934821 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a4d8d0-3044-4515-825d-06002bc0222c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:18.936367 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a4d8d0-3044-4515-825d-06002bc0222c None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:18.950354 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-b6a4d8d0-3044-4515-825d-06002bc0222c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:18.950354 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:18.950354 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:18.950354 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:18.950354 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:18.950354 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:18.950354 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:18.950354 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:18.950354 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:18.950354 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:18.950354 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:18.950354 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:18.950354 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:18.950354 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:18.950354 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:18.950354 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:18.951489 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:18.951489 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:18.951489 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:18.951489 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:18.951489 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:18.951489 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:18.951489 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:18.951489 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:18.951489 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:18.951489 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:18.951489 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:18.951489 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:18.951489 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:18.951489 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:18.951489 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:18.951489 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:18.951489 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:18.952572 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:18.952572 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:18.952572 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:18.955089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 285/579] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:18.955790 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4d8d0-3044-4515-825d-06002bc0222c None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:18.998741 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b056508e-a883-49e8-b359-af73fd782f07 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:19.023950 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:19.031922 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec177e2-418f-4a14-9db9-922a8e36fbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:19.032827 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec177e2-418f-4a14-9db9-922a8e36fbe8 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:19.033112 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec177e2-418f-4a14-9db9-922a8e36fbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:19.033381 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec177e2-418f-4a14-9db9-922a8e36fbe8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:19.040324 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec177e2-418f-4a14-9db9-922a8e36fbe8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:19.042294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec177e2-418f-4a14-9db9-922a8e36fbe8 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:19.053702 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-dec177e2-418f-4a14-9db9-922a8e36fbe8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 14:13:19.053702 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:19.053702 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:19.053702 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:19.053702 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:19.053702 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:19.053702 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:19.053702 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:19.053702 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:19.053702 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:19.053702 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:19.053702 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:19.053702 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:13:19.053702 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:13:19.053702 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:13:19.053702 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:13:19.060027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:19.060027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:19.060027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:13:19.060027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:19.060027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:13:19.060027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:19.060027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:19.060027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:19.060027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:19.060027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:19.060027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:19.060027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:19.060027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:19.060027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:19.060027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:19.060027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:19.060027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:19.060027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:19.061209 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:19.061209 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:19.061209 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:13:19.061209 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:13:19.061209 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:13:19.061209 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:13:19.061209 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:13:19.061209 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:13:19.061209 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 14:13:19.061209 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:19.061842 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4d8d0-3044-4515-825d-06002bc0222c None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:19.061928 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 295/580] 10.222.0.52 () {56 vars in 1093 bytes} [Sat Sep 24 14:13:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:19.061928 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec177e2-418f-4a14-9db9-922a8e36fbe8 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:19.093429 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492b37a1-8839-4898-b897-ba53fadb8c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:19.094029 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b37a1-8839-4898-b897-ba53fadb8c22 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:19.094303 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b37a1-8839-4898-b897-ba53fadb8c22 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:19.094572 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b37a1-8839-4898-b897-ba53fadb8c22 None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:19.095259 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492b37a1-8839-4898-b897-ba53fadb8c22 None admin] RBAC: Adding query filter params (name=service) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:19.095557 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492b37a1-8839-4898-b897-ba53fadb8c22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:19.096901 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492b37a1-8839-4898-b897-ba53fadb8c22 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:19.098065 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:19.105827 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 286/581] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 14:13:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:19.106953 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492b37a1-8839-4898-b897-ba53fadb8c22 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:19.158088 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec177e2-418f-4a14-9db9-922a8e36fbe8 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:19.191865 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd3375e-a371-4dc8-b2db-6f9ef39c7d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:19.192459 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3375e-a371-4dc8-b2db-6f9ef39c7d21 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:19.192736 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3375e-a371-4dc8-b2db-6f9ef39c7d21 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:19.193006 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3375e-a371-4dc8-b2db-6f9ef39c7d21 None admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:19.193682 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd3375e-a371-4dc8-b2db-6f9ef39c7d21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:19.193987 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd3375e-a371-4dc8-b2db-6f9ef39c7d21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:19.195451 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd3375e-a371-4dc8-b2db-6f9ef39c7d21 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:19.203892 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492b37a1-8839-4898-b897-ba53fadb8c22 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:19.207093 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 296/582] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 14:13:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:19.207878 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd3375e-a371-4dc8-b2db-6f9ef39c7d21 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:19.237292 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed6be8c-9eec-49b8-97bf-e599129a333c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:19.237891 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6be8c-9eec-49b8-97bf-e599129a333c None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:19.238170 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6be8c-9eec-49b8-97bf-e599129a333c None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:19.238439 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6be8c-9eec-49b8-97bf-e599129a333c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:19.239194 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed6be8c-9eec-49b8-97bf-e599129a333c None admin] RBAC: Adding query filter params (role.id=be958e9954f04268a9f012c775fc17e5, scope.project.id=36a3ae7bab25471fb989136150b6438c, user.id=879f296733114f6f844a98aaefe35e92) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:19.239503 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed6be8c-9eec-49b8-97bf-e599129a333c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:19.240922 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed6be8c-9eec-49b8-97bf-e599129a333c None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:19.242617 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:19.247505 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 287/583] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:13:19 2022] GET /identity/v3/role_assignments?user.id=879f296733114f6f844a98aaefe35e92&scope.project.id=36a3ae7bab25471fb989136150b6438c&role.id=be958e9954f04268a9f012c775fc17e5 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:19.248286 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed6be8c-9eec-49b8-97bf-e599129a333c None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:19.328938 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd3375e-a371-4dc8-b2db-6f9ef39c7d21 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:19.330690 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:19.362288 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed6be8c-9eec-49b8-97bf-e599129a333c None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:19.367814 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39480ab9-0ae1-4b9c-8d86-49cc12f01201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:19.368443 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39480ab9-0ae1-4b9c-8d86-49cc12f01201 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:19.368757 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39480ab9-0ae1-4b9c-8d86-49cc12f01201 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:19.369030 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39480ab9-0ae1-4b9c-8d86-49cc12f01201 None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:19.369683 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39480ab9-0ae1-4b9c-8d86-49cc12f01201 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:19.369986 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39480ab9-0ae1-4b9c-8d86-49cc12f01201 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:19.371454 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39480ab9-0ae1-4b9c-8d86-49cc12f01201 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:19.388056 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 297/584] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:19.388056 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39480ab9-0ae1-4b9c-8d86-49cc12f01201 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:19.405580 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2522660a-f93a-4efd-8e1c-a2d4a1516340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:19.406312 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522660a-f93a-4efd-8e1c-a2d4a1516340 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:19.406598 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522660a-f93a-4efd-8e1c-a2d4a1516340 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:19.406907 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522660a-f93a-4efd-8e1c-a2d4a1516340 None admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:19.407913 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2522660a-f93a-4efd-8e1c-a2d4a1516340 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:19.408274 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2522660a-f93a-4efd-8e1c-a2d4a1516340 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:19.409658 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2522660a-f93a-4efd-8e1c-a2d4a1516340 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:19.430843 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 288/585] 10.222.0.52 () {56 vars in 1133 bytes} [Sat Sep 24 14:13:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:19.430843 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2522660a-f93a-4efd-8e1c-a2d4a1516340 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:19.489349 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:19.501848 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39480ab9-0ae1-4b9c-8d86-49cc12f01201 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:19.537627 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2522660a-f93a-4efd-8e1c-a2d4a1516340 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:19.538584 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eb9c90-ac46-44f5-a027-9d0f2d766cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:19.539240 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb9c90-ac46-44f5-a027-9d0f2d766cec None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:19.539512 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb9c90-ac46-44f5-a027-9d0f2d766cec None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:19.539785 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb9c90-ac46-44f5-a027-9d0f2d766cec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:19.551579 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eb9c90-ac46-44f5-a027-9d0f2d766cec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:19.553059 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eb9c90-ac46-44f5-a027-9d0f2d766cec None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:19.567405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-f0eb9c90-ac46-44f5-a027-9d0f2d766cec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:19.567405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:19.567405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:19.567405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:19.567405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:19.567405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:19.567405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:19.567405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:19.567405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:19.567405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:19.567405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:19.567405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:19.567405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:19.567405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:19.567405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:19.567405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:19.568520 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:19.568520 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:19.568520 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:19.568520 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:19.568520 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:19.568520 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:19.568520 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:19.568520 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:19.568520 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:19.568520 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:19.568520 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:19.568520 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:19.568520 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:19.568520 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:19.568520 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:19.568520 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:19.568520 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:19.569589 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:19.569589 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:19.569589 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:19.570947 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 298/586] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:19.571902 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eb9c90-ac46-44f5-a027-9d0f2d766cec None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:19.579759 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f80f80-0f21-4934-a9ff-61ab86ffb2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:19.580746 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f80f80-0f21-4934-a9ff-61ab86ffb2ae None admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:19.581032 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f80f80-0f21-4934-a9ff-61ab86ffb2ae None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:19.581312 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f80f80-0f21-4934-a9ff-61ab86ffb2ae None admin] PATH_INFO: `/v3/projects/36a3ae7bab25471fb989136150b6438c/users/e82b01dbd6e84a9d82baa2c28d77fc28/roles/be958e9954f04268a9f012c775fc17e5` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:19.588172 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:19.602927 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f80f80-0f21-4934-a9ff-61ab86ffb2ae None admin] RBAC: Authorizing `identity:create_grant(project_id=36a3ae7bab25471fb989136150b6438c, user_id=e82b01dbd6e84a9d82baa2c28d77fc28, role_id=be958e9954f04268a9f012c775fc17e5)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:19.605089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f80f80-0f21-4934-a9ff-61ab86ffb2ae None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:19.622189 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 289/587] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 14:13:19 2022] PUT /identity/v3/projects/36a3ae7bab25471fb989136150b6438c/users/e82b01dbd6e84a9d82baa2c28d77fc28/roles/be958e9954f04268a9f012c775fc17e5 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:13:19.623087 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f80f80-0f21-4934-a9ff-61ab86ffb2ae None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:19.695186 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eb9c90-ac46-44f5-a027-9d0f2d766cec None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:19.745836 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d9ff3c-57e9-4a4a-a302-b75545f10a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:19.746668 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9ff3c-57e9-4a4a-a302-b75545f10a12 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:19.747069 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9ff3c-57e9-4a4a-a302-b75545f10a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:19.747481 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9ff3c-57e9-4a4a-a302-b75545f10a12 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:19.757146 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d9ff3c-57e9-4a4a-a302-b75545f10a12 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:19.758973 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d9ff3c-57e9-4a4a-a302-b75545f10a12 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:19.769333 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f80f80-0f21-4934-a9ff-61ab86ffb2ae None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:19.772443 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-d2d9ff3c-57e9-4a4a-a302-b75545f10a12 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 24 14:13:19.772443 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:19.772443 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:19.772443 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:19.772443 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:19.772443 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:19.772443 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:19.772443 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:19.772443 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:19.772443 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:19.772443 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:19.772443 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:19.772443 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:13:19.772443 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:13:19.772443 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:13:19.772443 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:13:19.773457 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:19.773457 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:19.773457 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:13:19.773457 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:19.773457 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:13:19.773457 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:19.773457 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:19.773457 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:19.773457 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:19.773457 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:19.773457 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:19.773457 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:19.773457 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:19.773457 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:19.773457 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:19.773457 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:19.773457 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:19.773457 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:19.774365 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:19.774365 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:19.774365 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:13:19.774365 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:13:19.774365 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:13:19.774365 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:13:19.774365 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:13:19.774365 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:13:19.774365 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 24 14:13:19.774365 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:19.777689 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 299/588] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:13:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:19.778793 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d9ff3c-57e9-4a4a-a302-b75545f10a12 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:19.783989 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:19.820655 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65413110-b517-4bae-97ef-5819047edc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:19.821317 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65413110-b517-4bae-97ef-5819047edc44 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:19.821600 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65413110-b517-4bae-97ef-5819047edc44 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:19.822411 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65413110-b517-4bae-97ef-5819047edc44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:19.843200 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65413110-b517-4bae-97ef-5819047edc44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:19.844754 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65413110-b517-4bae-97ef-5819047edc44 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:19.861053 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-65413110-b517-4bae-97ef-5819047edc44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:19.861053 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:19.861053 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:19.861053 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:19.861053 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:19.861053 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:19.861053 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:19.861053 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:19.861053 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:19.861053 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:19.861053 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:19.861053 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:19.861053 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:19.861053 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:19.861053 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:19.861053 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:19.862048 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:19.862048 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:19.862048 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:19.862048 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:19.862048 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:19.862048 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:19.862048 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:19.862048 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:19.862048 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:19.862048 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:19.862048 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:19.862048 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:19.862048 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:19.862048 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:19.862048 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:19.862048 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:19.862048 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:19.863025 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:19.863025 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:19.863025 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:19.863025 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 290/589] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:19.863025 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65413110-b517-4bae-97ef-5819047edc44 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:19.890366 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:19.920100 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d9ff3c-57e9-4a4a-a302-b75545f10a12 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:19.975237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556b2c0-1a7b-434b-b5b0-18c1b90c060c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:19.975863 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556b2c0-1a7b-434b-b5b0-18c1b90c060c None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:19.976263 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556b2c0-1a7b-434b-b5b0-18c1b90c060c None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:19.976603 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556b2c0-1a7b-434b-b5b0-18c1b90c060c None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:19.977321 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b556b2c0-1a7b-434b-b5b0-18c1b90c060c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:19.977691 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b556b2c0-1a7b-434b-b5b0-18c1b90c060c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:19.979767 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b556b2c0-1a7b-434b-b5b0-18c1b90c060c None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:19.992865 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 300/590] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:19.994516 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556b2c0-1a7b-434b-b5b0-18c1b90c060c None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:20.000435 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65413110-b517-4bae-97ef-5819047edc44 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:20.045436 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ed87bf-da37-475f-b658-b05fe2ac99b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:20.046163 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed87bf-da37-475f-b658-b05fe2ac99b6 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:20.046514 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed87bf-da37-475f-b658-b05fe2ac99b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:20.046898 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed87bf-da37-475f-b658-b05fe2ac99b6 None admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:20.047790 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ed87bf-da37-475f-b658-b05fe2ac99b6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:20.048188 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ed87bf-da37-475f-b658-b05fe2ac99b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:20.049751 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ed87bf-da37-475f-b658-b05fe2ac99b6 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:20.069052 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 291/591] 10.222.0.52 () {56 vars in 1137 bytes} [Sat Sep 24 14:13:19 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:20.069970 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ed87bf-da37-475f-b658-b05fe2ac99b6 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:20.092076 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:20.106208 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556b2c0-1a7b-434b-b5b0-18c1b90c060c None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:20.147022 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536f1b8-c3ab-414e-a7b8-46ccdf1b5f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:20.147985 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536f1b8-c3ab-414e-a7b8-46ccdf1b5f2b None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:20.150001 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536f1b8-c3ab-414e-a7b8-46ccdf1b5f2b None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:20.150859 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536f1b8-c3ab-414e-a7b8-46ccdf1b5f2b None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:20.151922 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2536f1b8-c3ab-414e-a7b8-46ccdf1b5f2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:20.152297 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2536f1b8-c3ab-414e-a7b8-46ccdf1b5f2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:20.154767 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2536f1b8-c3ab-414e-a7b8-46ccdf1b5f2b None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:20.156605 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:20.172594 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 301/592] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:20.174048 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536f1b8-c3ab-414e-a7b8-46ccdf1b5f2b None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:20.207471 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ed87bf-da37-475f-b658-b05fe2ac99b6 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:20.255799 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c3c1e0-4275-4b88-be1d-63bccd8df71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:20.257139 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c3c1e0-4275-4b88-be1d-63bccd8df71a None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:20.257596 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c3c1e0-4275-4b88-be1d-63bccd8df71a None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:20.258229 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c3c1e0-4275-4b88-be1d-63bccd8df71a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:20.271440 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c3c1e0-4275-4b88-be1d-63bccd8df71a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:20.275340 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c3c1e0-4275-4b88-be1d-63bccd8df71a None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:20.297968 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536f1b8-c3ab-414e-a7b8-46ccdf1b5f2b None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:20.300277 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-16c3c1e0-4275-4b88-be1d-63bccd8df71a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 24 14:13:20.300277 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:20.300277 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:20.300277 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:20.300277 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:20.300277 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:20.300277 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:20.300277 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:20.300277 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:20.300277 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:20.300277 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:20.300277 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:20.300277 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:13:20.300277 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:13:20.300277 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:13:20.300277 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:13:20.301183 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:20.301183 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:20.301183 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:13:20.301183 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:20.301183 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:13:20.301183 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:20.301183 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:20.301183 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:20.301183 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:20.301183 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:20.301183 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:20.301183 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:20.301183 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:20.301183 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:20.301183 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:20.301183 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:20.301183 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:20.301183 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:20.302145 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:20.302145 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:20.302145 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:13:20.302145 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:13:20.302145 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:13:20.302145 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:13:20.302145 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:13:20.302145 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:13:20.302145 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 24 14:13:20.302145 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:20.305958 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 292/593] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 14:13:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:20.307860 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c3c1e0-4275-4b88-be1d-63bccd8df71a None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:20.341581 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5316f187-ce2f-493e-9dd5-de0bc193a240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:20.342374 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316f187-ce2f-493e-9dd5-de0bc193a240 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:20.342796 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316f187-ce2f-493e-9dd5-de0bc193a240 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:20.343179 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316f187-ce2f-493e-9dd5-de0bc193a240 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:20.354808 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5316f187-ce2f-493e-9dd5-de0bc193a240 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:20.356478 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5316f187-ce2f-493e-9dd5-de0bc193a240 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:20.372058 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-5316f187-ce2f-493e-9dd5-de0bc193a240 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:20.372058 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:20.372058 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:20.372058 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:20.372058 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:20.372058 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:20.372058 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:20.372058 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:20.372058 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:20.372058 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:20.372058 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:20.372058 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:20.372058 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:20.372058 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:20.372058 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:20.372058 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:20.373234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:20.373234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:20.373234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:20.373234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:20.373234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:20.373234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:20.373234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:20.373234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:20.373234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:20.373234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:20.373234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:20.373234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:20.373234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:20.373234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:20.373234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:20.373234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:20.373234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:20.374331 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:20.374331 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:20.374331 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:20.375776 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 302/594] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:20.376678 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5316f187-ce2f-493e-9dd5-de0bc193a240 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:20.416568 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:20.431167 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:20.453655 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c3c1e0-4275-4b88-be1d-63bccd8df71a None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:20.487890 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5316f187-ce2f-493e-9dd5-de0bc193a240 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:20.501148 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6145725-e0fd-481b-89d6-b4e79ad936e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:20.502009 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6145725-e0fd-481b-89d6-b4e79ad936e3 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:20.502428 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6145725-e0fd-481b-89d6-b4e79ad936e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:20.503235 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6145725-e0fd-481b-89d6-b4e79ad936e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:20.513528 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6145725-e0fd-481b-89d6-b4e79ad936e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:20.515874 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6145725-e0fd-481b-89d6-b4e79ad936e3 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:20.525245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b618137-7f1b-4f8d-b170-a5f6cb489275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:20.525991 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b618137-7f1b-4f8d-b170-a5f6cb489275 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:20.526376 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b618137-7f1b-4f8d-b170-a5f6cb489275 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:20.526740 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b618137-7f1b-4f8d-b170-a5f6cb489275 None admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:20.527635 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b618137-7f1b-4f8d-b170-a5f6cb489275 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:20.528037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b618137-7f1b-4f8d-b170-a5f6cb489275 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:20.529775 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b618137-7f1b-4f8d-b170-a5f6cb489275 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:20.532011 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-e6145725-e0fd-481b-89d6-b4e79ad936e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:20.532011 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:20.532011 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:20.532011 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:20.532011 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:20.532011 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:20.532011 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:20.532011 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:20.532011 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:20.532011 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:20.532011 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:20.532011 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:20.532011 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:13:20.532011 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:13:20.532011 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:13:20.533295 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:13:20.533295 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:20.533295 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:20.533295 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:20.533295 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:20.533295 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:20.533295 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:20.533295 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:20.533295 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:20.533295 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:20.533295 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:20.533295 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:20.533295 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:20.533295 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:20.533295 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:20.533295 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:20.533295 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:20.534378 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:13:20.534378 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:13:20.534378 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:13:20.534378 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:13:20.534378 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:13:20.534378 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:13:20.534378 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:20.534378 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:20.537312 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 293/595] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:13:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:20.538403 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6145725-e0fd-481b-89d6-b4e79ad936e3 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:20.552275 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 303/596] 10.222.0.52 () {56 vars in 1139 bytes} [Sat Sep 24 14:13:20 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:20.553434 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b618137-7f1b-4f8d-b170-a5f6cb489275 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:20.663624 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b618137-7f1b-4f8d-b170-a5f6cb489275 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:20.667703 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:20.686458 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6145725-e0fd-481b-89d6-b4e79ad936e3 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:20.701198 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba305471-dc0e-46ba-ab49-5b190a160042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:20.701814 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba305471-dc0e-46ba-ab49-5b190a160042 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:20.702093 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba305471-dc0e-46ba-ab49-5b190a160042 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:20.702367 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba305471-dc0e-46ba-ab49-5b190a160042 None admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:20.703098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba305471-dc0e-46ba-ab49-5b190a160042 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:20.703402 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba305471-dc0e-46ba-ab49-5b190a160042 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:20.704834 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba305471-dc0e-46ba-ab49-5b190a160042 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:20.716814 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 304/597] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 14:13:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:20.717742 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba305471-dc0e-46ba-ab49-5b190a160042 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:20.729775 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac21d58b-f913-42bd-9f98-f33af204f74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:20.731071 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac21d58b-f913-42bd-9f98-f33af204f74d None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:20.731818 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac21d58b-f913-42bd-9f98-f33af204f74d None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:20.732283 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac21d58b-f913-42bd-9f98-f33af204f74d None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:20.733269 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac21d58b-f913-42bd-9f98-f33af204f74d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:20.733783 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac21d58b-f913-42bd-9f98-f33af204f74d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:20.735971 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac21d58b-f913-42bd-9f98-f33af204f74d None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:20.738946 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:20.755028 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 294/598] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:20.755725 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac21d58b-f913-42bd-9f98-f33af204f74d None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:20.838009 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba305471-dc0e-46ba-ab49-5b190a160042 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:20.874109 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77feb7ef-545c-43fa-9192-2b11473ecef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:20.874748 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77feb7ef-545c-43fa-9192-2b11473ecef6 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:20.875084 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77feb7ef-545c-43fa-9192-2b11473ecef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:20.875368 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77feb7ef-545c-43fa-9192-2b11473ecef6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:20.879507 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac21d58b-f913-42bd-9f98-f33af204f74d None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:20.886787 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77feb7ef-545c-43fa-9192-2b11473ecef6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:20.888270 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77feb7ef-545c-43fa-9192-2b11473ecef6 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:20.904212 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-77feb7ef-545c-43fa-9192-2b11473ecef6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:20.904212 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:20.904212 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:20.904212 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:20.904212 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:20.904212 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:20.904212 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:20.904212 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:20.904212 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:20.904212 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:20.904212 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:20.904212 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:20.904212 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:20.904212 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:20.904212 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:20.904212 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:20.906260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:20.906260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:20.906260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:20.906260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:20.906260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:20.906260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:20.906260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:20.906260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:20.906260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:20.906260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:20.906260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:20.906260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:20.906260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:20.906260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:20.906260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:20.906260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:20.906260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:20.908241 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:20.908241 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:20.908241 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:20.908241 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 305/599] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:20.908241 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77feb7ef-545c-43fa-9192-2b11473ecef6 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:20.925542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc9b45-23d5-416d-a9da-2f84c4c2e5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:20.926428 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:20.927126 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc9b45-23d5-416d-a9da-2f84c4c2e5fb None admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:20.928226 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc9b45-23d5-416d-a9da-2f84c4c2e5fb None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:20.928695 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc9b45-23d5-416d-a9da-2f84c4c2e5fb None admin] PATH_INFO: `/v3/projects/36a3ae7bab25471fb989136150b6438c/users/79eaaae97aac4282a613939e5b768376/roles/be958e9954f04268a9f012c775fc17e5` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:20.947741 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc9b45-23d5-416d-a9da-2f84c4c2e5fb None admin] RBAC: Authorizing `identity:create_grant(project_id=36a3ae7bab25471fb989136150b6438c, user_id=79eaaae97aac4282a613939e5b768376, role_id=be958e9954f04268a9f012c775fc17e5)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:20.949625 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc9b45-23d5-416d-a9da-2f84c4c2e5fb None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:20.966966 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 295/600] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 14:13:20 2022] PUT /identity/v3/projects/36a3ae7bab25471fb989136150b6438c/users/79eaaae97aac4282a613939e5b768376/roles/be958e9954f04268a9f012c775fc17e5 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:13:20.968449 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc9b45-23d5-416d-a9da-2f84c4c2e5fb None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:21.029362 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77feb7ef-545c-43fa-9192-2b11473ecef6 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:21.046944 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:21.070620 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f815a624-4b6a-4c6a-bec2-b24230068de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:21.071362 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815a624-4b6a-4c6a-bec2-b24230068de4 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:21.071651 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815a624-4b6a-4c6a-bec2-b24230068de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:21.072494 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815a624-4b6a-4c6a-bec2-b24230068de4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:21.081399 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f815a624-4b6a-4c6a-bec2-b24230068de4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:21.083069 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f815a624-4b6a-4c6a-bec2-b24230068de4 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:21.096974 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-f815a624-4b6a-4c6a-bec2-b24230068de4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:21.096974 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:21.096974 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:21.096974 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:21.096974 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:21.096974 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:21.096974 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:21.096974 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:21.096974 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:21.096974 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:21.096974 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:21.096974 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:21.096974 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:13:21.096974 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:13:21.096974 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:13:21.097894 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:13:21.097894 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:21.097894 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:21.097894 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:21.097894 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:21.097894 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:21.097894 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:21.097894 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:21.097894 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:21.097894 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:21.097894 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:21.097894 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:21.097894 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:21.097894 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:21.097894 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:21.097894 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:21.097894 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:21.099901 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:13:21.099901 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:13:21.099901 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:13:21.099901 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:13:21.099901 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:13:21.099901 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:13:21.099901 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:21.099901 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:21.099901 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 306/601] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:13:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:21.104394 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f815a624-4b6a-4c6a-bec2-b24230068de4 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:21.120866 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc9b45-23d5-416d-a9da-2f84c4c2e5fb None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:21.158406 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c537368-840a-4d4d-a6b4-f5a1f64c4d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:21.159063 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c537368-840a-4d4d-a6b4-f5a1f64c4d01 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:21.159341 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c537368-840a-4d4d-a6b4-f5a1f64c4d01 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:21.159613 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c537368-840a-4d4d-a6b4-f5a1f64c4d01 None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:21.160276 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c537368-840a-4d4d-a6b4-f5a1f64c4d01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:21.160577 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c537368-840a-4d4d-a6b4-f5a1f64c4d01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:21.162074 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c537368-840a-4d4d-a6b4-f5a1f64c4d01 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:21.175680 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 296/602] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:21.181135 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c537368-840a-4d4d-a6b4-f5a1f64c4d01 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:21.211646 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:21.239488 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f815a624-4b6a-4c6a-bec2-b24230068de4 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:21.277678 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d85b5c8-c58c-474b-b680-416938580430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:21.278677 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85b5c8-c58c-474b-b680-416938580430 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:21.279176 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85b5c8-c58c-474b-b680-416938580430 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:21.279783 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85b5c8-c58c-474b-b680-416938580430 None admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:21.280663 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d85b5c8-c58c-474b-b680-416938580430 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:21.281157 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d85b5c8-c58c-474b-b680-416938580430 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:21.283304 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d85b5c8-c58c-474b-b680-416938580430 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:21.293168 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c537368-840a-4d4d-a6b4-f5a1f64c4d01 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:21.298499 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 307/603] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 14:13:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:21.305599 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d85b5c8-c58c-474b-b680-416938580430 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:21.307854 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:21.340484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ce8b6a-566a-4571-b41e-d49065616a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:21.341182 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce8b6a-566a-4571-b41e-d49065616a22 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:21.341462 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce8b6a-566a-4571-b41e-d49065616a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:21.341734 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce8b6a-566a-4571-b41e-d49065616a22 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:21.351112 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ce8b6a-566a-4571-b41e-d49065616a22 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:21.352497 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ce8b6a-566a-4571-b41e-d49065616a22 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:21.366748 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-c5ce8b6a-566a-4571-b41e-d49065616a22 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:21.366748 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:21.366748 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:21.366748 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:21.366748 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:21.366748 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:21.366748 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:21.366748 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:21.366748 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:21.366748 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:21.366748 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:21.366748 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:21.366748 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:13:21.366748 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:13:21.366748 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:13:21.367771 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:13:21.367771 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:21.367771 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:21.367771 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:21.367771 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:21.367771 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:21.367771 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:21.367771 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:21.367771 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:21.367771 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:21.367771 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:21.367771 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:21.367771 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:21.367771 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:21.367771 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:21.367771 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:21.367771 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:21.369915 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:13:21.369915 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:13:21.369915 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:13:21.369915 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:13:21.369915 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:13:21.369915 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:13:21.369915 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:21.369915 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:21.369915 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 297/604] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:13:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:21.375100 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ce8b6a-566a-4571-b41e-d49065616a22 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:21.427204 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d85b5c8-c58c-474b-b680-416938580430 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:21.464878 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f25c24-aff2-4888-9648-d679cfb189ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:21.465936 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f25c24-aff2-4888-9648-d679cfb189ca None admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:21.466447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f25c24-aff2-4888-9648-d679cfb189ca None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:21.467090 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f25c24-aff2-4888-9648-d679cfb189ca None admin] PATH_INFO: `/v3/projects/36a3ae7bab25471fb989136150b6438c/users/63921623056045a9b97b2de3ca418138/roles/be958e9954f04268a9f012c775fc17e5` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:21.470986 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:21.484967 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f25c24-aff2-4888-9648-d679cfb189ca None admin] RBAC: Authorizing `identity:create_grant(project_id=36a3ae7bab25471fb989136150b6438c, user_id=63921623056045a9b97b2de3ca418138, role_id=be958e9954f04268a9f012c775fc17e5)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:21.486604 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f25c24-aff2-4888-9648-d679cfb189ca None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:21.490970 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ce8b6a-566a-4571-b41e-d49065616a22 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:21.508809 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 308/605] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 14:13:21 2022] PUT /identity/v3/projects/36a3ae7bab25471fb989136150b6438c/users/63921623056045a9b97b2de3ca418138/roles/be958e9954f04268a9f012c775fc17e5 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:13:21.548865 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7eea90-f62e-41c3-8f19-2f2a30284ebd None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:21.549226 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7eea90-f62e-41c3-8f19-2f2a30284ebd None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:21.549483 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7eea90-f62e-41c3-8f19-2f2a30284ebd None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:21.551145 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 309/606] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:13:21.557381 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92ed03-ecbe-4fe2-aef3-6056784590ba None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:21.558092 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92ed03-ecbe-4fe2-aef3-6056784590ba None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:21.558580 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92ed03-ecbe-4fe2-aef3-6056784590ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:21.575235 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd0d4e-d070-4893-ba40-4672a208e2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:21.575859 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd0d4e-d070-4893-ba40-4672a208e2c5 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:21.577736 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd0d4e-d070-4893-ba40-4672a208e2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:21.577736 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd0d4e-d070-4893-ba40-4672a208e2c5 None admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:21.577736 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bd0d4e-d070-4893-ba40-4672a208e2c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:21.577736 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bd0d4e-d070-4893-ba40-4672a208e2c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:21.579150 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bd0d4e-d070-4893-ba40-4672a208e2c5 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:21.592813 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-fa92ed03-ecbe-4fe2-aef3-6056784590ba None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:21.595201 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-fa92ed03-ecbe-4fe2-aef3-6056784590ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:21.606870 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 298/607] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 14:13:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:21.613275 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd0d4e-d070-4893-ba40-4672a208e2c5 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:21.670756 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:21.721844 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 310/608] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:13:21.788429 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd0d4e-d070-4893-ba40-4672a208e2c5 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:21.839322 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f639d7ef-6309-4ba4-934f-627db42e19bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:21.839322 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639d7ef-6309-4ba4-934f-627db42e19bb None admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:21.839322 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639d7ef-6309-4ba4-934f-627db42e19bb None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:21.839322 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639d7ef-6309-4ba4-934f-627db42e19bb None admin] PATH_INFO: `/v3/projects/36a3ae7bab25471fb989136150b6438c/users/a9348164132846d2b593ddfa29a820eb/roles/be958e9954f04268a9f012c775fc17e5` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:21.859433 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f639d7ef-6309-4ba4-934f-627db42e19bb None admin] RBAC: Authorizing `identity:create_grant(project_id=36a3ae7bab25471fb989136150b6438c, user_id=a9348164132846d2b593ddfa29a820eb, role_id=be958e9954f04268a9f012c775fc17e5)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:21.860854 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f639d7ef-6309-4ba4-934f-627db42e19bb None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:21.877134 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 299/609] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 14:13:21 2022] PUT /identity/v3/projects/36a3ae7bab25471fb989136150b6438c/users/a9348164132846d2b593ddfa29a820eb/roles/be958e9954f04268a9f012c775fc17e5 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:13:22.076930 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa92ed03-ecbe-4fe2-aef3-6056784590ba None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:22.188569 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa92ed03-ecbe-4fe2-aef3-6056784590ba None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:22.211519 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:22.214717 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7fee5a-51ac-4ebe-ada4-9189449ce24d None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:22.215041 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7fee5a-51ac-4ebe-ada4-9189449ce24d None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:22.215326 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7fee5a-51ac-4ebe-ada4-9189449ce24d None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:22.216930 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 300/610] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:22.224242 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24c2ed-8d86-418a-acb3-0a0e36f19e16 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:22.224523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24c2ed-8d86-418a-acb3-0a0e36f19e16 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:22.224796 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24c2ed-8d86-418a-acb3-0a0e36f19e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:22.229730 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeefe754-f44c-42dd-bdaf-be26882956a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:22.230970 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefe754-f44c-42dd-bdaf-be26882956a1 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:22.231250 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefe754-f44c-42dd-bdaf-be26882956a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:22.231516 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefe754-f44c-42dd-bdaf-be26882956a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:22.245360 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeefe754-f44c-42dd-bdaf-be26882956a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:13:22.250238 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-eeefe754-f44c-42dd-bdaf-be26882956a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:13:22.250238 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:22.250238 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:22.250238 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:22.250238 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:22.250238 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:22.250238 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:22.250238 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:22.250238 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:22.250238 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:22.250238 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:22.250238 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:22.250238 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:13:22.250238 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:13:22.250238 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:13:22.250238 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:13:22.253195 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:13:22.253195 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:13:22.253195 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:13:22.253195 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:13:22.253195 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:22.253195 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:22.253195 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:22.253195 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:22.253195 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:22.253195 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:22.253195 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:22.253195 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:22.253195 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:22.253195 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:22.253195 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:22.253195 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:22.254037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:22.254037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:22.254037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:22.254037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:22.254037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:13:22.254037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:13:22.254037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:13:22.254037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:13:22.254037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:13:22.254037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:13:22.254037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:13:22.254037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:22.254037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 311/611] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 14:13:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:22.260571 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-cc24c2ed-8d86-418a-acb3-0a0e36f19e16 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:22.261615 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-cc24c2ed-8d86-418a-acb3-0a0e36f19e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:22.264169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeefe754-f44c-42dd-bdaf-be26882956a1 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:22.338847 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 301/612] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:13:22.391300 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeefe754-f44c-42dd-bdaf-be26882956a1 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:22.442358 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86f479e-a0b3-426b-8db6-ae1455d0e51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:22.442903 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f479e-a0b3-426b-8db6-ae1455d0e51e None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:22.443179 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f479e-a0b3-426b-8db6-ae1455d0e51e None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:22.443517 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f479e-a0b3-426b-8db6-ae1455d0e51e None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:22.444222 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86f479e-a0b3-426b-8db6-ae1455d0e51e None admin] RBAC: Adding query filter params (name=service) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:22.444570 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86f479e-a0b3-426b-8db6-ae1455d0e51e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:22.445928 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86f479e-a0b3-426b-8db6-ae1455d0e51e None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:22.458456 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 312/613] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 14:13:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:22.464259 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc24c2ed-8d86-418a-acb3-0a0e36f19e16 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:22.472885 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:22.574549 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc24c2ed-8d86-418a-acb3-0a0e36f19e16 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:22.609426 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee8735-aa2a-416f-a1c1-dea35d32690f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:22.610085 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee8735-aa2a-416f-a1c1-dea35d32690f None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:22.610358 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee8735-aa2a-416f-a1c1-dea35d32690f None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:22.610627 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee8735-aa2a-416f-a1c1-dea35d32690f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:22.621512 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ee8735-aa2a-416f-a1c1-dea35d32690f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:22.622978 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ee8735-aa2a-416f-a1c1-dea35d32690f None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:22.638458 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-75ee8735-aa2a-416f-a1c1-dea35d32690f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:22.638458 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:22.638458 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:22.638458 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:22.638458 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:22.638458 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:22.638458 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:22.638458 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:22.638458 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:22.638458 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:22.638458 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:22.638458 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:22.638458 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:22.638458 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:22.638458 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:22.638458 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:22.639441 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:22.639441 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:22.639441 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:22.639441 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:22.639441 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:22.639441 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:22.639441 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:22.639441 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:22.639441 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:22.639441 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:22.639441 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:22.639441 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:22.639441 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:22.639441 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:22.639441 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:22.639441 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:22.639441 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:22.640327 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:22.640327 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:22.640327 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:22.640327 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 302/614] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:22.648169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86f479e-a0b3-426b-8db6-ae1455d0e51e None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:22.691327 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:22.755286 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86f479e-a0b3-426b-8db6-ae1455d0e51e None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:22.765609 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee8735-aa2a-416f-a1c1-dea35d32690f None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:22.797186 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7deb93-30b5-4f72-847b-da4423c61a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:22.798403 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7deb93-30b5-4f72-847b-da4423c61a68 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:22.799214 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7deb93-30b5-4f72-847b-da4423c61a68 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:22.799980 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7deb93-30b5-4f72-847b-da4423c61a68 None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:22.800996 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7deb93-30b5-4f72-847b-da4423c61a68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:22.801512 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7deb93-30b5-4f72-847b-da4423c61a68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:22.803672 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7deb93-30b5-4f72-847b-da4423c61a68 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:22.826287 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 313/615] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:22.834697 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7deb93-30b5-4f72-847b-da4423c61a68 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:22.855414 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:22.947802 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee8735-aa2a-416f-a1c1-dea35d32690f None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:22.986412 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:22.996540 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7deb93-30b5-4f72-847b-da4423c61a68 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:22.999856 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2079f4-16f5-40b3-93b3-c1cd21512ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:23.000507 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2079f4-16f5-40b3-93b3-c1cd21512ded None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:23.007145 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2079f4-16f5-40b3-93b3-c1cd21512ded None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:23.007386 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2079f4-16f5-40b3-93b3-c1cd21512ded None admin] PATH_INFO: `/v3/roles/service` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:23.026065 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2079f4-16f5-40b3-93b3-c1cd21512ded None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:13:23.038330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-9f2079f4-16f5-40b3-93b3-c1cd21512ded None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:13:23.038330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:23.038330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:23.038330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:23.038330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:23.038330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:23.038330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:23.038330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:23.038330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:23.038330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:23.038330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:23.038330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:23.038330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:13:23.038330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:13:23.038330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:13:23.038330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:13:23.039300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:13:23.039300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:13:23.039300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:13:23.039300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:13:23.039300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:23.039300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:23.039300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:23.039300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:23.039300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:23.039300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:23.039300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:23.039300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:23.039300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:23.039300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:23.039300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:23.039300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:23.040137 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:23.040137 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:23.040137 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:23.040137 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:23.040137 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:13:23.040137 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:13:23.040137 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:13:23.040137 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:13:23.040137 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:13:23.040137 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:13:23.040137 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:13:23.040137 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:23.042391 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 303/616] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 14:13:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:13:23.047686 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2079f4-16f5-40b3-93b3-c1cd21512ded None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:23.063757 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe84c3b1-877f-4898-b9ed-bf2fa71747ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:23.064402 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84c3b1-877f-4898-b9ed-bf2fa71747ea None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:23.064682 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84c3b1-877f-4898-b9ed-bf2fa71747ea None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:23.064958 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84c3b1-877f-4898-b9ed-bf2fa71747ea None admin] PATH_INFO: `/v3/users/swift` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:23.080208 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe84c3b1-877f-4898-b9ed-bf2fa71747ea None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:23.082725 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe84c3b1-877f-4898-b9ed-bf2fa71747ea None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:23.095411 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-fe84c3b1-877f-4898-b9ed-bf2fa71747ea None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 24 14:13:23.095411 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:23.095411 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:23.095411 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:23.095411 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:23.095411 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:23.095411 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:23.095411 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:23.095411 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:23.095411 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:23.095411 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:23.095411 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:23.095411 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:13:23.095411 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:13:23.095411 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:13:23.095411 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:13:23.096344 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:23.096344 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:23.096344 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:13:23.096344 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:23.096344 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:13:23.096344 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:23.096344 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:23.096344 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:23.096344 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:23.096344 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:23.096344 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:23.096344 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:23.096344 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:23.096344 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:23.096344 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:23.096344 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:23.096344 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:23.096344 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:23.097272 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:23.097272 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:23.097272 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:13:23.097272 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:13:23.097272 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:13:23.097272 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:13:23.097272 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:13:23.097272 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:13:23.097272 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 24 14:13:23.097272 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:23.100807 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 314/617] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 14:13:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:23.102611 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe84c3b1-877f-4898-b9ed-bf2fa71747ea None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:23.248833 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2079f4-16f5-40b3-93b3-c1cd21512ded None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:23.275888 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe84c3b1-877f-4898-b9ed-bf2fa71747ea None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:23.293917 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:23.328392 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e503c-25ce-410b-b2cd-3953e6764a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:23.329446 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e503c-25ce-410b-b2cd-3953e6764a4e None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:23.329968 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e503c-25ce-410b-b2cd-3953e6764a4e None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:23.330446 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e503c-25ce-410b-b2cd-3953e6764a4e None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:23.331338 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70e503c-25ce-410b-b2cd-3953e6764a4e None admin] RBAC: Adding query filter params (name=service) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:23.331847 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70e503c-25ce-410b-b2cd-3953e6764a4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:23.333401 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70e503c-25ce-410b-b2cd-3953e6764a4e None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:23.334615 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b7971-0a1f-4712-b15b-7106ab4738f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:23.335360 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b7971-0a1f-4712-b15b-7106ab4738f2 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:23.335648 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b7971-0a1f-4712-b15b-7106ab4738f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:23.335924 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b7971-0a1f-4712-b15b-7106ab4738f2 None admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:23.341704 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142b7971-0a1f-4712-b15b-7106ab4738f2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:23.341704 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142b7971-0a1f-4712-b15b-7106ab4738f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:23.341704 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142b7971-0a1f-4712-b15b-7106ab4738f2 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:23.349873 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 304/618] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 14:13:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:23.355639 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e503c-25ce-410b-b2cd-3953e6764a4e None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:23.365385 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 315/619] 10.222.0.52 () {56 vars in 1135 bytes} [Sat Sep 24 14:13:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:23.366637 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b7971-0a1f-4712-b15b-7106ab4738f2 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:23.390561 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:23.482049 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b7971-0a1f-4712-b15b-7106ab4738f2 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:23.527200 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b07385b-b132-4bf5-8c81-880c1f4c656a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:23.528233 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07385b-b132-4bf5-8c81-880c1f4c656a None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:23.528656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07385b-b132-4bf5-8c81-880c1f4c656a None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:23.528940 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07385b-b132-4bf5-8c81-880c1f4c656a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:23.541104 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b07385b-b132-4bf5-8c81-880c1f4c656a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:23.542734 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b07385b-b132-4bf5-8c81-880c1f4c656a None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:23.560048 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-8b07385b-b132-4bf5-8c81-880c1f4c656a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:23.560048 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:23.560048 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:23.560048 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:23.560048 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:23.560048 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:23.560048 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:23.560048 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:23.560048 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:23.560048 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:23.560048 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:23.560048 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:23.560048 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:23.560048 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:23.560048 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:23.560048 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:23.561037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:23.561037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:23.561037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:23.561037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:23.561037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:23.561037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:23.561037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:23.561037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:23.561037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:23.561037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:23.561037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:23.561037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:23.561037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:23.561037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:23.561037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:23.561037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:23.561037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:23.561901 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:23.561901 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:23.561901 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:23.566246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 316/620] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:23.567718 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b07385b-b132-4bf5-8c81-880c1f4c656a None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:23.610559 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e503c-25ce-410b-b2cd-3953e6764a4e None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:23.648862 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebaff2c-c249-4bce-afa8-2654357a82f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:23.649501 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebaff2c-c249-4bce-afa8-2654357a82f2 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:23.649779 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebaff2c-c249-4bce-afa8-2654357a82f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:23.650087 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebaff2c-c249-4bce-afa8-2654357a82f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:23.672145 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebaff2c-c249-4bce-afa8-2654357a82f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:23.673601 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebaff2c-c249-4bce-afa8-2654357a82f2 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:23.689063 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:23.703806 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-2ebaff2c-c249-4bce-afa8-2654357a82f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:23.703806 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:23.703806 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:23.703806 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:23.703806 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:23.703806 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:23.703806 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:23.703806 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:23.703806 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:23.703806 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:23.703806 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:23.703806 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:23.703806 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:23.703806 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:23.703806 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:23.703806 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:23.704757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:23.704757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:23.704757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:23.704757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:23.704757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:23.704757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:23.704757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:23.704757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:23.704757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:23.704757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:23.704757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:23.704757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:23.704757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:23.704757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:23.704757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:23.704757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:23.704757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:23.705618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:23.705618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:23.705618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:23.705618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 305/621] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:13:23.713397 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebaff2c-c249-4bce-afa8-2654357a82f2 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:23.752615 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b07385b-b132-4bf5-8c81-880c1f4c656a None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:23.787309 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:23.812753 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e52580d-fd8e-40b4-964d-5bb4c1aad210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:23.813374 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52580d-fd8e-40b4-964d-5bb4c1aad210 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:23.813654 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52580d-fd8e-40b4-964d-5bb4c1aad210 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:23.813932 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52580d-fd8e-40b4-964d-5bb4c1aad210 None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:23.815342 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e52580d-fd8e-40b4-964d-5bb4c1aad210 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:23.815655 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e52580d-fd8e-40b4-964d-5bb4c1aad210 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:23.817058 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e52580d-fd8e-40b4-964d-5bb4c1aad210 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:23.844853 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 317/622] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:13:23.847750 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e52580d-fd8e-40b4-964d-5bb4c1aad210 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:23.922153 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebaff2c-c249-4bce-afa8-2654357a82f2 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:23.956755 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da319c7-03dc-4c75-9ce2-52d1242be4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:23.957371 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da319c7-03dc-4c75-9ce2-52d1242be4ed None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:23.957650 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da319c7-03dc-4c75-9ce2-52d1242be4ed None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:23.957921 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da319c7-03dc-4c75-9ce2-52d1242be4ed None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:23.958618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da319c7-03dc-4c75-9ce2-52d1242be4ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:23.958952 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da319c7-03dc-4c75-9ce2-52d1242be4ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:23.960343 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da319c7-03dc-4c75-9ce2-52d1242be4ed None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:23.974190 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 306/623] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:13:23.979656 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c2524-30a4-4235-b353-d334be7a57e1 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:23.980009 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c2524-30a4-4235-b353-d334be7a57e1 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:23.980282 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c2524-30a4-4235-b353-d334be7a57e1 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:23.985782 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 307/624] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:23.985782 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c2524-30a4-4235-b353-d334be7a57e1 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:23.994626 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e52580d-fd8e-40b4-964d-5bb4c1aad210 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:24.022912 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:24.032881 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82450dd9-c128-44a5-abad-6b1cf7876f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:24.033857 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82450dd9-c128-44a5-abad-6b1cf7876f55 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:24.034461 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82450dd9-c128-44a5-abad-6b1cf7876f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:24.034923 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82450dd9-c128-44a5-abad-6b1cf7876f55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:24.042663 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82450dd9-c128-44a5-abad-6b1cf7876f55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:24.044502 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82450dd9-c128-44a5-abad-6b1cf7876f55 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:24.056686 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-82450dd9-c128-44a5-abad-6b1cf7876f55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:24.056686 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:24.056686 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:24.056686 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:24.056686 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:24.056686 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:24.056686 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:24.056686 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:24.056686 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:24.056686 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:24.056686 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:24.056686 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:24.056686 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:13:24.056686 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:13:24.056686 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:13:24.057749 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:13:24.057749 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:24.057749 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:24.057749 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:24.057749 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:24.057749 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:24.057749 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:24.057749 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:24.057749 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:24.057749 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:24.057749 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:24.057749 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:24.057749 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:24.057749 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:24.057749 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:24.057749 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:24.057749 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:24.058797 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:13:24.058797 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:13:24.058797 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:13:24.058797 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:13:24.058797 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:13:24.058797 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:13:24.058797 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:24.058797 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:24.060871 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 318/625] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:13:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:24.062198 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af04217e-e935-4778-a657-c8585df56728 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:24.062610 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af04217e-e935-4778-a657-c8585df56728 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:24.063163 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af04217e-e935-4778-a657-c8585df56728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:24.081045 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:24.082376 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-af04217e-e935-4778-a657-c8585df56728 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:24.083664 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-af04217e-e935-4778-a657-c8585df56728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:24.092346 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c2524-30a4-4235-b353-d334be7a57e1 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:24.126786 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b67bde3-3ef4-4f83-97bb-043ad64adb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:24.127449 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67bde3-3ef4-4f83-97bb-043ad64adb30 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:24.127728 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67bde3-3ef4-4f83-97bb-043ad64adb30 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:24.128000 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67bde3-3ef4-4f83-97bb-043ad64adb30 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:24.135424 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b67bde3-3ef4-4f83-97bb-043ad64adb30 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:24.137146 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b67bde3-3ef4-4f83-97bb-043ad64adb30 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:24.150536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-9b67bde3-3ef4-4f83-97bb-043ad64adb30 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 24 14:13:24.150536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:24.150536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:24.150536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:24.150536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:24.150536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:24.150536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:24.150536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:24.150536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:24.150536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:24.150536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:24.150536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:24.150536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:13:24.150536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:13:24.150536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:13:24.150536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:13:24.151688 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:24.151688 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:24.151688 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:13:24.151688 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:24.151688 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:13:24.151688 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:24.151688 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:24.151688 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:24.151688 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:24.151688 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:24.151688 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:24.151688 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:24.151688 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:24.151688 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:24.151688 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:24.151688 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:24.151688 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:24.151688 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:24.152847 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:24.152847 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:24.152847 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:13:24.152847 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:13:24.152847 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:13:24.152847 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:13:24.152847 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:13:24.152847 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:13:24.152847 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 24 14:13:24.152847 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:24.162837 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 308/626] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:13:23 2022] GET /identity/v3/users/glance => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:24.162837 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc5027-bb51-4d22-8909-3e9bf997bc1c None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:24.162837 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc5027-bb51-4d22-8909-3e9bf997bc1c None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:24.162837 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc5027-bb51-4d22-8909-3e9bf997bc1c None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:24.162837 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 309/627] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:24.162837 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc5027-bb51-4d22-8909-3e9bf997bc1c None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:24.163293 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 319/628] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:13:24.166839 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af04217e-e935-4778-a657-c8585df56728 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:24.273828 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc5027-bb51-4d22-8909-3e9bf997bc1c None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:24.296178 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af04217e-e935-4778-a657-c8585df56728 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:24.314277 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db987ff-3088-4bcd-9aa2-943d4140cbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:24.314684 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:24.315585 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db987ff-3088-4bcd-9aa2-943d4140cbf2 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:24.315842 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db987ff-3088-4bcd-9aa2-943d4140cbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:24.316117 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db987ff-3088-4bcd-9aa2-943d4140cbf2 None admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:24.316774 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db987ff-3088-4bcd-9aa2-943d4140cbf2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:24.317076 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db987ff-3088-4bcd-9aa2-943d4140cbf2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:24.318501 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db987ff-3088-4bcd-9aa2-943d4140cbf2 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:24.334955 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 310/629] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 14:13:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:24.336544 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173276d9-0ad7-496f-97ef-9fd652507132 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:24.337017 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173276d9-0ad7-496f-97ef-9fd652507132 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:24.338028 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173276d9-0ad7-496f-97ef-9fd652507132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:24.343355 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5676476-ef69-416f-988b-c6a0886759dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:24.344268 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5676476-ef69-416f-988b-c6a0886759dd None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:24.344736 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5676476-ef69-416f-988b-c6a0886759dd None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:24.345267 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5676476-ef69-416f-988b-c6a0886759dd None admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:24.346237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5676476-ef69-416f-988b-c6a0886759dd None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:24.346781 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5676476-ef69-416f-988b-c6a0886759dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:24.350590 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5676476-ef69-416f-988b-c6a0886759dd None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:24.365854 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-173276d9-0ad7-496f-97ef-9fd652507132 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:24.367172 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-173276d9-0ad7-496f-97ef-9fd652507132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:24.371696 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 320/630] 10.222.0.52 () {56 vars in 1137 bytes} [Sat Sep 24 14:13:24 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:24.372771 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5676476-ef69-416f-988b-c6a0886759dd None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:24.377319 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:24.449092 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 311/631] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:13:24.449937 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173276d9-0ad7-496f-97ef-9fd652507132 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:24.477848 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5676476-ef69-416f-988b-c6a0886759dd None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:24.514036 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a60ead-cfdf-4b89-9d75-615a092c8eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:24.515603 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a60ead-cfdf-4b89-9d75-615a092c8eb9 None admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:24.516138 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a60ead-cfdf-4b89-9d75-615a092c8eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:24.516614 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a60ead-cfdf-4b89-9d75-615a092c8eb9 None admin] PATH_INFO: `/v3/projects/36a3ae7bab25471fb989136150b6438c/users/879f296733114f6f844a98aaefe35e92/roles/be958e9954f04268a9f012c775fc17e5` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:24.532159 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a60ead-cfdf-4b89-9d75-615a092c8eb9 None admin] RBAC: Authorizing `identity:create_grant(project_id=36a3ae7bab25471fb989136150b6438c, user_id=879f296733114f6f844a98aaefe35e92, role_id=be958e9954f04268a9f012c775fc17e5)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:24.534135 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a60ead-cfdf-4b89-9d75-615a092c8eb9 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:24.550384 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 321/632] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 14:13:24 2022] PUT /identity/v3/projects/36a3ae7bab25471fb989136150b6438c/users/879f296733114f6f844a98aaefe35e92/roles/be958e9954f04268a9f012c775fc17e5 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:13:24.551559 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a60ead-cfdf-4b89-9d75-615a092c8eb9 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:24.583476 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:24.583817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173276d9-0ad7-496f-97ef-9fd652507132 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:24.671634 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23829399-1750-4c92-89d7-3ca2fccc657a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:24.672275 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23829399-1750-4c92-89d7-3ca2fccc657a None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:24.672553 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23829399-1750-4c92-89d7-3ca2fccc657a None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:24.672959 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23829399-1750-4c92-89d7-3ca2fccc657a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:24.676047 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a60ead-cfdf-4b89-9d75-615a092c8eb9 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:24.695537 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:24.718670 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5afa5c-f200-4c10-be2f-c4950a8f57f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:24.719820 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5afa5c-f200-4c10-be2f-c4950a8f57f3 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:24.720349 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5afa5c-f200-4c10-be2f-c4950a8f57f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:24.720927 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5afa5c-f200-4c10-be2f-c4950a8f57f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:24.734586 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5afa5c-f200-4c10-be2f-c4950a8f57f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:13:24.741225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-4f5afa5c-f200-4c10-be2f-c4950a8f57f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:13:24.741225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:24.741225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:24.741225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:24.741225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:24.741225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:24.741225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:24.741225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:24.741225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:24.741225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:24.741225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:24.741225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:24.741225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:13:24.741225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:13:24.741225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:13:24.741225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:13:24.742182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:13:24.742182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:13:24.742182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:13:24.742182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:13:24.742182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:24.742182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:24.742182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:24.742182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:24.742182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:24.742182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:24.742182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:24.742182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:24.742182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:24.742182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:24.742182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:24.742182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:24.743057 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:24.743057 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:24.743057 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:24.743057 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:24.743057 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:13:24.743057 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:13:24.743057 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:13:24.743057 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:13:24.743057 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:13:24.743057 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:13:24.743057 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:13:24.743057 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:24.746619 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 322/633] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 14:13:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:24.747951 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5afa5c-f200-4c10-be2f-c4950a8f57f3 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:24.785904 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23829399-1750-4c92-89d7-3ca2fccc657a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:24.787427 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23829399-1750-4c92-89d7-3ca2fccc657a None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:24.858571 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:24.876292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5afa5c-f200-4c10-be2f-c4950a8f57f3 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:24.907244 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 312/634] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:13:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:13:24.908331 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23829399-1750-4c92-89d7-3ca2fccc657a None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:24.913531 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de8bf3-8cc7-44a8-be51-ec1839007176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:24.914164 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de8bf3-8cc7-44a8-be51-ec1839007176 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:24.914441 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de8bf3-8cc7-44a8-be51-ec1839007176 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:24.914708 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de8bf3-8cc7-44a8-be51-ec1839007176 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:24.925606 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de8bf3-8cc7-44a8-be51-ec1839007176 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:13:24.930446 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-e6de8bf3-8cc7-44a8-be51-ec1839007176 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:13:24.930446 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:24.930446 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:24.930446 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:24.930446 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:24.930446 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:24.930446 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:24.930446 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:24.930446 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:24.930446 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:24.930446 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:24.930446 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:24.930446 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:13:24.930446 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:13:24.930446 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:13:24.930446 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:13:24.932207 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:13:24.932207 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:13:24.932207 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:13:24.932207 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:13:24.932207 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:24.932207 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:24.932207 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:24.932207 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:24.932207 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:24.932207 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:24.932207 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:24.932207 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:24.932207 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:24.932207 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:24.932207 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:24.932207 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:24.933396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:24.933396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:24.933396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:24.933396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:24.933396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:13:24.933396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:13:24.933396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:13:24.933396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:13:24.933396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:13:24.933396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:13:24.933396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:13:24.933396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:24.933396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 323/635] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 14:13:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:24.934253 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99efb51-e8c9-457b-b9cb-2c171b9ba2f2 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:24.934253 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99efb51-e8c9-457b-b9cb-2c171b9ba2f2 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:24.934253 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99efb51-e8c9-457b-b9cb-2c171b9ba2f2 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:24.935357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 324/636] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:24.936041 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99efb51-e8c9-457b-b9cb-2c171b9ba2f2 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:24.963205 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:25.027925 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23829399-1750-4c92-89d7-3ca2fccc657a None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:25.052914 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99efb51-e8c9-457b-b9cb-2c171b9ba2f2 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:25.067200 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ba9ff-7a6d-4368-b0a6-598750ede516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:25.067812 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ba9ff-7a6d-4368-b0a6-598750ede516 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:25.068085 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ba9ff-7a6d-4368-b0a6-598750ede516 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:25.068559 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ba9ff-7a6d-4368-b0a6-598750ede516 None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:25.069008 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2ba9ff-7a6d-4368-b0a6-598750ede516 None admin] RBAC: Adding query filter params (name=service) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:25.069309 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2ba9ff-7a6d-4368-b0a6-598750ede516 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:25.070656 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2ba9ff-7a6d-4368-b0a6-598750ede516 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:25.079064 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 313/637] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 14:13:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:25.079937 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ba9ff-7a6d-4368-b0a6-598750ede516 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:25.094975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08920cc6-02bc-4307-9015-df29d907a50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:25.096030 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08920cc6-02bc-4307-9015-df29d907a50c None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:25.096515 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08920cc6-02bc-4307-9015-df29d907a50c None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:25.096959 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08920cc6-02bc-4307-9015-df29d907a50c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:25.108784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08920cc6-02bc-4307-9015-df29d907a50c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:25.110653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08920cc6-02bc-4307-9015-df29d907a50c None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:25.118697 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:25.128628 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-08920cc6-02bc-4307-9015-df29d907a50c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:25.128628 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:25.128628 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:25.128628 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:25.128628 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:25.128628 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:25.128628 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:25.128628 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:25.128628 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:25.128628 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:25.128628 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:25.128628 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:25.128628 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:25.128628 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:25.128628 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:25.128628 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:25.129808 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:25.129808 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:25.129808 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:25.129808 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:25.129808 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:25.129808 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:25.129808 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:25.129808 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:25.129808 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:25.129808 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:25.129808 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:25.129808 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:25.129808 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:25.129808 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:25.129808 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:25.129808 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:25.129808 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:25.130983 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:25.130983 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:25.130983 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:25.136236 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 325/638] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:25.137528 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d92f34a-8e0e-48bc-938c-83da0095ec13 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:25.137813 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d92f34a-8e0e-48bc-938c-83da0095ec13 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:25.138135 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d92f34a-8e0e-48bc-938c-83da0095ec13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:25.156422 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-9d92f34a-8e0e-48bc-938c-83da0095ec13 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:25.157843 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-9d92f34a-8e0e-48bc-938c-83da0095ec13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:25.188086 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ba9ff-7a6d-4368-b0a6-598750ede516 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:25.191391 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:25.224577 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8cae67-48d4-4dc8-80f2-dbdb89d96e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:25.225182 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8cae67-48d4-4dc8-80f2-dbdb89d96e4c None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:25.225464 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8cae67-48d4-4dc8-80f2-dbdb89d96e4c None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:25.225737 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8cae67-48d4-4dc8-80f2-dbdb89d96e4c None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:25.226388 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8cae67-48d4-4dc8-80f2-dbdb89d96e4c None admin] RBAC: Adding query filter params (name=service) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:25.226687 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8cae67-48d4-4dc8-80f2-dbdb89d96e4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:25.228768 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8cae67-48d4-4dc8-80f2-dbdb89d96e4c None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:25.230654 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 326/639] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:13:25.241570 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 314/640] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 14:13:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:25.241570 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8cae67-48d4-4dc8-80f2-dbdb89d96e4c None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:25.241806 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d92f34a-8e0e-48bc-938c-83da0095ec13 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:25.359331 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8cae67-48d4-4dc8-80f2-dbdb89d96e4c None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:25.371330 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d92f34a-8e0e-48bc-938c-83da0095ec13 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:25.400095 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:25.410443 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4daacc7-f7eb-4577-8027-9e2724ca9390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:25.411126 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4daacc7-f7eb-4577-8027-9e2724ca9390 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:25.411402 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4daacc7-f7eb-4577-8027-9e2724ca9390 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:25.411645 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f357d371-2a83-45d3-b279-7c5f4c9961bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:25.412089 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4daacc7-f7eb-4577-8027-9e2724ca9390 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:25.413015 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357d371-2a83-45d3-b279-7c5f4c9961bb None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:25.413498 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357d371-2a83-45d3-b279-7c5f4c9961bb None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:25.414174 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357d371-2a83-45d3-b279-7c5f4c9961bb None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:25.417388 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f357d371-2a83-45d3-b279-7c5f4c9961bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:25.417703 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f357d371-2a83-45d3-b279-7c5f4c9961bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:25.419162 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f357d371-2a83-45d3-b279-7c5f4c9961bb None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:25.424916 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4daacc7-f7eb-4577-8027-9e2724ca9390 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:25.427176 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4daacc7-f7eb-4577-8027-9e2724ca9390 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:25.432530 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 315/641] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:25.433356 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f357d371-2a83-45d3-b279-7c5f4c9961bb None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:25.448794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-d4daacc7-f7eb-4577-8027-9e2724ca9390 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:25.448794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:25.448794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:25.448794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:25.448794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:25.448794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:25.448794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:25.448794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:25.448794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:25.448794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:25.448794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:25.448794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:25.448794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:25.448794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:25.448794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:25.448794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:25.449765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:25.449765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:25.449765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:25.449765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:25.449765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:25.449765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:25.449765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:25.449765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:25.449765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:25.449765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:25.449765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:25.449765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:25.449765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:25.449765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:25.449765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:25.449765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:25.449765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:25.450632 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:25.450632 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:25.450632 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:25.457579 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 327/642] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:25.457579 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4daacc7-f7eb-4577-8027-9e2724ca9390 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:25.469087 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:25.551154 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f357d371-2a83-45d3-b279-7c5f4c9961bb None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:25.576503 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4daacc7-f7eb-4577-8027-9e2724ca9390 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:25.589618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6d1e94-3276-4d97-9a0f-2a1d0af412c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:25.590476 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d1e94-3276-4d97-9a0f-2a1d0af412c9 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:25.590752 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d1e94-3276-4d97-9a0f-2a1d0af412c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:25.591068 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d1e94-3276-4d97-9a0f-2a1d0af412c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:25.601680 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6d1e94-3276-4d97-9a0f-2a1d0af412c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:25.603270 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6d1e94-3276-4d97-9a0f-2a1d0af412c9 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:25.609325 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df13a65f-b41d-489c-a039-e717debedc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:25.610008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13a65f-b41d-489c-a039-e717debedc58 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:25.610238 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13a65f-b41d-489c-a039-e717debedc58 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:25.610506 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13a65f-b41d-489c-a039-e717debedc58 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:25.617096 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df13a65f-b41d-489c-a039-e717debedc58 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:25.618509 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-5d6d1e94-3276-4d97-9a0f-2a1d0af412c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:25.618509 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:25.618509 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:25.618509 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:25.618509 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:25.618509 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:25.618509 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:25.618509 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:25.618509 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:25.618509 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:25.618509 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:25.618509 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:25.618509 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:25.618509 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:25.618509 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:25.618509 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:25.620051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:25.620051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:25.620051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:25.620051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:25.620051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:25.620051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:25.620051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:25.620051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:25.620051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:25.620051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:25.620051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:25.620051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:25.620051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:25.620051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:25.620051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:25.620051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:25.620051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:25.620942 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:25.620942 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:25.620942 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:25.621112 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df13a65f-b41d-489c-a039-e717debedc58 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:25.621193 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 316/643] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:25.621267 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6d1e94-3276-4d97-9a0f-2a1d0af412c9 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:25.637172 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-df13a65f-b41d-489c-a039-e717debedc58 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:25.637172 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:25.637172 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:25.637172 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:25.637172 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:25.637172 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:25.637172 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:25.637172 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:25.637172 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:25.637172 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:25.637172 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:25.637172 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:25.637172 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:13:25.637172 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:13:25.637172 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:13:25.638066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:13:25.638066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:25.638066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:25.638066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:25.638066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:25.638066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:25.638066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:25.638066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:25.638066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:25.638066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:25.638066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:25.638066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:25.638066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:25.638066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:25.638066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:25.638066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:25.638066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:25.639074 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:13:25.639074 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:13:25.639074 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:13:25.639074 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:13:25.639074 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:13:25.639074 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:13:25.639074 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:25.639074 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:25.639074 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 328/644] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:13:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:25.639074 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df13a65f-b41d-489c-a039-e717debedc58 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:25.686238 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:25.714974 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:25.740847 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6d1e94-3276-4d97-9a0f-2a1d0af412c9 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:25.750320 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df13a65f-b41d-489c-a039-e717debedc58 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:25.777993 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acdd2c2-8b1f-4640-9583-359108e0dffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:25.778600 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdd2c2-8b1f-4640-9583-359108e0dffc None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:25.778911 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdd2c2-8b1f-4640-9583-359108e0dffc None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:25.779188 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdd2c2-8b1f-4640-9583-359108e0dffc None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:25.779825 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acdd2c2-8b1f-4640-9583-359108e0dffc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:25.780127 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acdd2c2-8b1f-4640-9583-359108e0dffc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:25.781503 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acdd2c2-8b1f-4640-9583-359108e0dffc None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:25.784461 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7307eb63-77f6-4164-b1a5-5f81b5767937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:25.785089 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307eb63-77f6-4164-b1a5-5f81b5767937 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:25.785365 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307eb63-77f6-4164-b1a5-5f81b5767937 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:25.785634 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307eb63-77f6-4164-b1a5-5f81b5767937 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:25.795349 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 317/645] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:25.796671 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7307eb63-77f6-4164-b1a5-5f81b5767937 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:13:25.800396 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acdd2c2-8b1f-4640-9583-359108e0dffc None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:25.801822 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-7307eb63-77f6-4164-b1a5-5f81b5767937 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:13:25.801822 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:25.801822 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:25.801822 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:25.801822 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:25.801822 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:25.801822 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:25.801822 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:25.801822 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:25.801822 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:25.801822 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:25.801822 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:25.801822 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:13:25.801822 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:13:25.801822 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:13:25.801822 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:13:25.802717 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:13:25.802717 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:13:25.802717 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:13:25.802717 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:13:25.802717 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:25.802717 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:25.802717 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:25.802717 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:25.802717 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:25.802717 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:25.802717 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:25.802717 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:25.802717 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:25.802717 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:25.802717 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:25.802717 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:25.803574 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:25.803574 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:25.803574 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:25.803574 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:25.803574 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:13:25.803574 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:13:25.803574 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:13:25.803574 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:13:25.803574 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:13:25.803574 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:13:25.803574 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:13:25.803574 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:25.803574 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 329/646] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 14:13:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:25.804259 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7307eb63-77f6-4164-b1a5-5f81b5767937 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:25.904287 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:25.907226 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7307eb63-77f6-4164-b1a5-5f81b5767937 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:25.918869 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acdd2c2-8b1f-4640-9583-359108e0dffc None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:25.941169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee8e26d-d2db-4426-9a1f-c43b3c8e74fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:25.941774 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8e26d-d2db-4426-9a1f-c43b3c8e74fb None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:25.942047 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8e26d-d2db-4426-9a1f-c43b3c8e74fb None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:25.942314 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8e26d-d2db-4426-9a1f-c43b3c8e74fb None admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:25.943020 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee8e26d-d2db-4426-9a1f-c43b3c8e74fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:25.943320 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee8e26d-d2db-4426-9a1f-c43b3c8e74fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:25.944753 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee8e26d-d2db-4426-9a1f-c43b3c8e74fb None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:25.956326 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 330/647] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 14:13:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:25.956570 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398052c2-48e9-48e5-9ed0-1ce56e509aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:25.957114 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee8e26d-d2db-4426-9a1f-c43b3c8e74fb None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:25.957591 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398052c2-48e9-48e5-9ed0-1ce56e509aef None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:25.957868 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398052c2-48e9-48e5-9ed0-1ce56e509aef None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:25.958130 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398052c2-48e9-48e5-9ed0-1ce56e509aef None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:25.958786 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398052c2-48e9-48e5-9ed0-1ce56e509aef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:25.959116 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398052c2-48e9-48e5-9ed0-1ce56e509aef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:25.960497 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398052c2-48e9-48e5-9ed0-1ce56e509aef None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:25.968058 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:25.979243 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 318/648] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:25.979243 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398052c2-48e9-48e5-9ed0-1ce56e509aef None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:26.073838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee8e26d-d2db-4426-9a1f-c43b3c8e74fb None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:26.079493 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398052c2-48e9-48e5-9ed0-1ce56e509aef None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:26.112145 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c317b8ec-0ced-45de-8eb7-744d67a1ad0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:26.112767 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317b8ec-0ced-45de-8eb7-744d67a1ad0b None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:26.113145 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317b8ec-0ced-45de-8eb7-744d67a1ad0b None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:26.113512 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317b8ec-0ced-45de-8eb7-744d67a1ad0b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:26.114326 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906f24a-2cac-4c40-9b05-6035b3cbbeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:26.114970 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906f24a-2cac-4c40-9b05-6035b3cbbeb0 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:26.115369 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906f24a-2cac-4c40-9b05-6035b3cbbeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:26.115644 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906f24a-2cac-4c40-9b05-6035b3cbbeb0 None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:26.116311 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a906f24a-2cac-4c40-9b05-6035b3cbbeb0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:26.116612 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a906f24a-2cac-4c40-9b05-6035b3cbbeb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:26.117961 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a906f24a-2cac-4c40-9b05-6035b3cbbeb0 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:26.121497 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c317b8ec-0ced-45de-8eb7-744d67a1ad0b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:26.123585 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c317b8ec-0ced-45de-8eb7-744d67a1ad0b None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:26.125948 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 319/649] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 14:13:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:26.126657 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906f24a-2cac-4c40-9b05-6035b3cbbeb0 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:26.141069 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-c317b8ec-0ced-45de-8eb7-744d67a1ad0b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 14:13:26.141069 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:26.141069 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:26.141069 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:26.141069 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:26.141069 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:26.141069 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:26.141069 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:26.141069 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:26.141069 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:26.141069 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:26.141069 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:26.141069 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:13:26.141069 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:13:26.141069 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:13:26.141069 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:13:26.142019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:26.142019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:26.142019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:13:26.142019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:26.142019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:13:26.142019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:26.142019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:26.142019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:26.142019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:26.142019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:26.142019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:26.142019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:26.142019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:26.142019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:26.142019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:26.142019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:26.142019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:26.142019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:26.142975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:26.142975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:26.142975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:13:26.142975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:13:26.142975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:13:26.142975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:13:26.142975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:13:26.142975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:13:26.142975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 14:13:26.142975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:26.142975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 331/650] 10.222.0.52 () {56 vars in 1093 bytes} [Sat Sep 24 14:13:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:26.142975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c317b8ec-0ced-45de-8eb7-744d67a1ad0b None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:26.165437 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:26.197637 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:26.240068 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906f24a-2cac-4c40-9b05-6035b3cbbeb0 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:26.257265 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c317b8ec-0ced-45de-8eb7-744d67a1ad0b None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:26.278694 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf41b94-d2d3-40ba-a9df-003e63759b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:26.279544 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf41b94-d2d3-40ba-a9df-003e63759b83 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:26.279967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf41b94-d2d3-40ba-a9df-003e63759b83 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:26.280512 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf41b94-d2d3-40ba-a9df-003e63759b83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:26.281835 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf41b94-d2d3-40ba-a9df-003e63759b83 None admin] RBAC: Adding query filter params (role.id=be958e9954f04268a9f012c775fc17e5, scope.project.id=36a3ae7bab25471fb989136150b6438c, user.id=e82b01dbd6e84a9d82baa2c28d77fc28) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:26.282389 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf41b94-d2d3-40ba-a9df-003e63759b83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:26.284243 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf41b94-d2d3-40ba-a9df-003e63759b83 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:26.290933 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 320/651] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:13:26 2022] GET /identity/v3/role_assignments?user.id=e82b01dbd6e84a9d82baa2c28d77fc28&scope.project.id=36a3ae7bab25471fb989136150b6438c&role.id=be958e9954f04268a9f012c775fc17e5 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:26.291760 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf41b94-d2d3-40ba-a9df-003e63759b83 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:26.299911 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0a0d1c-7451-423b-8085-da9a79ec163f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:26.300859 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a0d1c-7451-423b-8085-da9a79ec163f None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:26.301354 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a0d1c-7451-423b-8085-da9a79ec163f None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:26.301812 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a0d1c-7451-423b-8085-da9a79ec163f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:26.311019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0a0d1c-7451-423b-8085-da9a79ec163f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:26.312809 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0a0d1c-7451-423b-8085-da9a79ec163f None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:26.326149 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-5f0a0d1c-7451-423b-8085-da9a79ec163f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 24 14:13:26.326149 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:26.326149 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:26.326149 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:26.326149 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:26.326149 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:26.326149 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:26.326149 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:26.326149 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:26.326149 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:26.326149 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:26.326149 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:26.326149 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:13:26.326149 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:13:26.326149 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:13:26.326149 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:13:26.327184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:26.327184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:26.327184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:13:26.327184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:26.327184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:13:26.327184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:26.327184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:26.327184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:26.327184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:26.327184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:26.327184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:26.327184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:26.327184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:26.327184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:26.327184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:26.327184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:26.327184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:26.327184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:26.328108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:26.328108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:26.328108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:13:26.328108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:13:26.328108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:13:26.328108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:13:26.328108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:13:26.328108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:13:26.328108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 24 14:13:26.328108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:26.331094 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 332/652] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:13:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:26.332275 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0a0d1c-7451-423b-8085-da9a79ec163f None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:26.440815 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf41b94-d2d3-40ba-a9df-003e63759b83 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:26.449057 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:26.475501 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0a0d1c-7451-423b-8085-da9a79ec163f None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:26.478856 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:26.493756 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e718cc0-792e-4008-afac-90fed643ab98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:26.495660 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e718cc0-792e-4008-afac-90fed643ab98 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:26.495660 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e718cc0-792e-4008-afac-90fed643ab98 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:26.495660 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e718cc0-792e-4008-afac-90fed643ab98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:26.510453 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e718cc0-792e-4008-afac-90fed643ab98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:26.512555 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e718cc0-792e-4008-afac-90fed643ab98 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:26.524799 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ecba3b-4ce4-4635-99f7-be046b86c1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:26.525986 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecba3b-4ce4-4635-99f7-be046b86c1dd None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:26.526248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecba3b-4ce4-4635-99f7-be046b86c1dd None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:26.526511 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecba3b-4ce4-4635-99f7-be046b86c1dd None admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:26.527342 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ecba3b-4ce4-4635-99f7-be046b86c1dd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:26.527672 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ecba3b-4ce4-4635-99f7-be046b86c1dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:26.529156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ecba3b-4ce4-4635-99f7-be046b86c1dd None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:26.531303 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-6e718cc0-792e-4008-afac-90fed643ab98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:26.531303 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:26.531303 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:26.531303 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:26.531303 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:26.531303 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:26.531303 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:26.531303 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:26.531303 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:26.531303 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:26.531303 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:26.531303 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:26.531303 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:26.531303 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:26.531303 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:26.531303 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:26.532202 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:26.532202 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:26.532202 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:26.532202 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:26.532202 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:26.532202 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:26.532202 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:26.532202 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:26.532202 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:26.532202 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:26.532202 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:26.532202 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:26.532202 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:26.532202 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:26.532202 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:26.532202 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:26.532202 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:26.533723 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:26.533723 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:26.533723 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:26.533723 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 321/653] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:26.533723 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e718cc0-792e-4008-afac-90fed643ab98 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:26.552675 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 333/654] 10.222.0.52 () {56 vars in 1133 bytes} [Sat Sep 24 14:13:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:26.553519 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ecba3b-4ce4-4635-99f7-be046b86c1dd None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:26.672463 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e718cc0-792e-4008-afac-90fed643ab98 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:26.676753 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ecba3b-4ce4-4635-99f7-be046b86c1dd None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:26.713517 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d56e58f-af3d-414f-a62e-375e610f7a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:26.714129 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56e58f-af3d-414f-a62e-375e610f7a3c None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:26.714451 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56e58f-af3d-414f-a62e-375e610f7a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:26.714934 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56e58f-af3d-414f-a62e-375e610f7a3c None admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:26.715725 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d56e58f-af3d-414f-a62e-375e610f7a3c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:26.716092 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d56e58f-af3d-414f-a62e-375e610f7a3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:26.717709 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d56e58f-af3d-414f-a62e-375e610f7a3c None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:26.728531 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6afb4d8-d831-4425-b034-92029082111c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:26.730640 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6afb4d8-d831-4425-b034-92029082111c None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:26.731187 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6afb4d8-d831-4425-b034-92029082111c None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:26.731734 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6afb4d8-d831-4425-b034-92029082111c None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:26.732662 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6afb4d8-d831-4425-b034-92029082111c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:26.733175 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6afb4d8-d831-4425-b034-92029082111c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:26.734830 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6afb4d8-d831-4425-b034-92029082111c None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:26.739833 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 322/655] 10.222.0.52 () {56 vars in 1137 bytes} [Sat Sep 24 14:13:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:26.740905 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d56e58f-af3d-414f-a62e-375e610f7a3c None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:26.747554 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:26.758094 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 334/656] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:26.760306 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6afb4d8-d831-4425-b034-92029082111c None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:26.782210 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:26.869127 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d56e58f-af3d-414f-a62e-375e610f7a3c None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:26.906617 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9555988-7c7f-4cc9-8225-7ce7eeac9529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:26.907267 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9555988-7c7f-4cc9-8225-7ce7eeac9529 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:26.907550 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9555988-7c7f-4cc9-8225-7ce7eeac9529 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:26.908078 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9555988-7c7f-4cc9-8225-7ce7eeac9529 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:26.908651 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6afb4d8-d831-4425-b034-92029082111c None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:26.945546 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a181db-cf00-4714-9ff1-63e33c98889c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:26.946155 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a181db-cf00-4714-9ff1-63e33c98889c None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:26.946437 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a181db-cf00-4714-9ff1-63e33c98889c None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:26.946708 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a181db-cf00-4714-9ff1-63e33c98889c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:26.985225 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9555988-7c7f-4cc9-8225-7ce7eeac9529 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:26.986598 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9555988-7c7f-4cc9-8225-7ce7eeac9529 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:27.016895 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:27.022374 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a181db-cf00-4714-9ff1-63e33c98889c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:27.023879 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a181db-cf00-4714-9ff1-63e33c98889c None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:27.024939 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:27.110394 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 323/657] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:13:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:13:27.115648 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9555988-7c7f-4cc9-8225-7ce7eeac9529 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:27.139256 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 335/658] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:13:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:13:27.140931 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54a467-4eab-40af-a784-025b8f94a228 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:27.141235 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54a467-4eab-40af-a784-025b8f94a228 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:27.141511 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54a467-4eab-40af-a784-025b8f94a228 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:27.145016 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 336/659] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:27.145016 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba54a467-4eab-40af-a784-025b8f94a228 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:27.219814 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9555988-7c7f-4cc9-8225-7ce7eeac9529 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:27.245511 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:27.250594 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba54a467-4eab-40af-a784-025b8f94a228 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:27.258068 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f0cbda-8b37-415f-813a-43b1261d0bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:27.258941 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f0cbda-8b37-415f-813a-43b1261d0bb8 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:27.259419 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f0cbda-8b37-415f-813a-43b1261d0bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:27.259838 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f0cbda-8b37-415f-813a-43b1261d0bb8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:27.261940 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:27.269326 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f0cbda-8b37-415f-813a-43b1261d0bb8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:27.270959 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f0cbda-8b37-415f-813a-43b1261d0bb8 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:27.284451 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-42f0cbda-8b37-415f-813a-43b1261d0bb8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 24 14:13:27.284451 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:27.284451 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:27.284451 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:27.284451 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:27.284451 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:27.284451 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:27.284451 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:27.284451 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:27.284451 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:27.284451 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:27.284451 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:27.284451 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:13:27.284451 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:13:27.284451 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:13:27.284451 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:13:27.285567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:27.285567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:27.285567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:13:27.285567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:27.285567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:13:27.285567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:27.285567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:27.285567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:27.285567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:27.285567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:27.285567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:27.285567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:27.285567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:27.285567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:27.285567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:27.285567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:27.285567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:27.285567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:27.286665 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:27.286665 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:27.286665 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:13:27.286665 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:13:27.286665 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:13:27.286665 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:13:27.286665 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:13:27.286665 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:13:27.286665 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 24 14:13:27.286665 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:27.287383 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3791b8e-fcca-45dd-aa4c-5f6b4f5f732d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:27.287666 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3791b8e-fcca-45dd-aa4c-5f6b4f5f732d None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:27.287941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3791b8e-fcca-45dd-aa4c-5f6b4f5f732d None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:27.288207 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3791b8e-fcca-45dd-aa4c-5f6b4f5f732d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:27.289089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 324/660] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 14:13:27 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:27.289994 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f0cbda-8b37-415f-813a-43b1261d0bb8 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:27.303441 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3791b8e-fcca-45dd-aa4c-5f6b4f5f732d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:27.305209 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3791b8e-fcca-45dd-aa4c-5f6b4f5f732d None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:27.321494 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-a3791b8e-fcca-45dd-aa4c-5f6b4f5f732d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:27.321494 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:27.321494 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:27.321494 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:27.321494 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:27.321494 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:27.321494 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:27.321494 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:27.321494 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:27.321494 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:27.321494 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:27.321494 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:27.321494 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:27.321494 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:27.321494 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:27.321494 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:27.322743 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:27.322743 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:27.322743 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:27.322743 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:27.322743 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:27.322743 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:27.322743 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:27.322743 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:27.322743 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:27.322743 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:27.322743 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:27.322743 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:27.322743 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:27.322743 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:27.322743 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:27.322743 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:27.322743 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:27.323915 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:27.323915 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:27.323915 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:27.324580 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 337/661] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:27.325647 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8068659-5f4c-4f35-ae40-b708f9d55ce1 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:27.325925 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8068659-5f4c-4f35-ae40-b708f9d55ce1 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:27.326196 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8068659-5f4c-4f35-ae40-b708f9d55ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:27.345540 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-c8068659-5f4c-4f35-ae40-b708f9d55ce1 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:27.346731 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-c8068659-5f4c-4f35-ae40-b708f9d55ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:27.411090 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f0cbda-8b37-415f-813a-43b1261d0bb8 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:27.419045 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 338/662] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:13:27.427361 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8068659-5f4c-4f35-ae40-b708f9d55ce1 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:27.454137 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af749a-5795-43cc-9a85-e51ed898e78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:27.455783 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af749a-5795-43cc-9a85-e51ed898e78e None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:27.456317 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af749a-5795-43cc-9a85-e51ed898e78e None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:27.456739 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af749a-5795-43cc-9a85-e51ed898e78e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:27.471265 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af749a-5795-43cc-9a85-e51ed898e78e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:27.472916 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af749a-5795-43cc-9a85-e51ed898e78e None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:27.488867 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-d3af749a-5795-43cc-9a85-e51ed898e78e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:27.488867 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:27.488867 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:27.488867 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:27.488867 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:27.488867 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:27.488867 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:27.488867 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:27.488867 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:27.488867 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:27.488867 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:27.488867 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:27.488867 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:27.488867 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:27.488867 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:27.488867 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:27.490098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:27.490098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:27.490098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:27.490098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:27.490098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:27.490098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:27.490098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:27.490098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:27.490098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:27.490098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:27.490098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:27.490098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:27.490098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:27.490098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:27.490098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:27.490098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:27.490098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:27.491226 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:27.491226 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:27.491226 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:27.496487 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 325/663] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:27.497252 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af749a-5795-43cc-9a85-e51ed898e78e None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:27.515993 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:27.537865 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8068659-5f4c-4f35-ae40-b708f9d55ce1 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:27.552102 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:27.576699 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6984f27d-1409-46c4-8ca3-49d7876418d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:27.577687 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6984f27d-1409-46c4-8ca3-49d7876418d5 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:27.578208 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6984f27d-1409-46c4-8ca3-49d7876418d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:27.578655 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6984f27d-1409-46c4-8ca3-49d7876418d5 None admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:27.579644 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6984f27d-1409-46c4-8ca3-49d7876418d5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:27.580126 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6984f27d-1409-46c4-8ca3-49d7876418d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:27.581741 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6984f27d-1409-46c4-8ca3-49d7876418d5 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:27.600941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 339/664] 10.222.0.52 () {56 vars in 1139 bytes} [Sat Sep 24 14:13:27 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:27.602148 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6984f27d-1409-46c4-8ca3-49d7876418d5 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:27.608922 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af749a-5795-43cc-9a85-e51ed898e78e None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:27.645785 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136dccf0-ee2f-4ba5-9090-64bca4415ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:27.646389 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136dccf0-ee2f-4ba5-9090-64bca4415ff1 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:27.646665 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136dccf0-ee2f-4ba5-9090-64bca4415ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:27.646965 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136dccf0-ee2f-4ba5-9090-64bca4415ff1 None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:27.647602 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136dccf0-ee2f-4ba5-9090-64bca4415ff1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:27.647897 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136dccf0-ee2f-4ba5-9090-64bca4415ff1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:27.649249 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136dccf0-ee2f-4ba5-9090-64bca4415ff1 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:27.662847 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 326/665] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:27.662847 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136dccf0-ee2f-4ba5-9090-64bca4415ff1 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:27.732604 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6984f27d-1409-46c4-8ca3-49d7876418d5 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:27.766714 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64847a8-970a-4650-a69b-4e5bbccb78aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:27.767263 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64847a8-970a-4650-a69b-4e5bbccb78aa None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:27.767537 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64847a8-970a-4650-a69b-4e5bbccb78aa None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:27.767801 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64847a8-970a-4650-a69b-4e5bbccb78aa None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:27.768435 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64847a8-970a-4650-a69b-4e5bbccb78aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:27.768732 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64847a8-970a-4650-a69b-4e5bbccb78aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:27.770089 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64847a8-970a-4650-a69b-4e5bbccb78aa None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:27.774588 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136dccf0-ee2f-4ba5-9090-64bca4415ff1 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:27.776986 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:27.784199 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:27.784345 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 340/666] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:27.785072 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64847a8-970a-4650-a69b-4e5bbccb78aa None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:27.815757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4282c52b-2e84-4091-ba0b-2d6f919cce34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:27.816360 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4282c52b-2e84-4091-ba0b-2d6f919cce34 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:27.816635 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4282c52b-2e84-4091-ba0b-2d6f919cce34 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:27.816901 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4282c52b-2e84-4091-ba0b-2d6f919cce34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:27.887406 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4282c52b-2e84-4091-ba0b-2d6f919cce34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:27.888850 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4282c52b-2e84-4091-ba0b-2d6f919cce34 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:27.896421 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64847a8-970a-4650-a69b-4e5bbccb78aa None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:27.932283 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bd281b-4912-4d48-a92f-fdde3c059d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:27.932928 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd281b-4912-4d48-a92f-fdde3c059d5c None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:27.933253 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd281b-4912-4d48-a92f-fdde3c059d5c None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:27.933521 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd281b-4912-4d48-a92f-fdde3c059d5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:27.940605 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bd281b-4912-4d48-a92f-fdde3c059d5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:27.942080 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bd281b-4912-4d48-a92f-fdde3c059d5c None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:27.954472 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-86bd281b-4912-4d48-a92f-fdde3c059d5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:27.954472 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:27.954472 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:27.954472 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:27.954472 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:27.954472 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:27.954472 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:27.954472 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:27.954472 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:27.954472 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:27.954472 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:27.954472 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:27.954472 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:13:27.954472 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:13:27.954472 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:13:27.955382 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:13:27.955382 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:27.955382 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:27.955382 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:27.955382 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:27.955382 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:27.955382 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:27.955382 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:27.955382 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:27.955382 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:27.955382 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:27.955382 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:27.955382 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:27.955382 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:27.955382 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:27.955382 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:27.955382 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:27.956256 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:13:27.956256 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:13:27.956256 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:13:27.956256 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:13:27.956256 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:13:27.956256 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:13:27.956256 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:27.956256 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:27.956256 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 341/667] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:13:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:27.956727 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bd281b-4912-4d48-a92f-fdde3c059d5c None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:27.993211 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 327/668] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:13:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:13:27.994078 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4282c52b-2e84-4091-ba0b-2d6f919cce34 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:28.001947 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:28.022533 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:28.067394 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bd281b-4912-4d48-a92f-fdde3c059d5c None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:28.099158 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4282c52b-2e84-4091-ba0b-2d6f919cce34 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:28.102010 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6e973-cd8f-4eb2-a18b-fdd73ac23f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:28.102143 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6e973-cd8f-4eb2-a18b-fdd73ac23f40 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:28.102396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6e973-cd8f-4eb2-a18b-fdd73ac23f40 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:28.102661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6e973-cd8f-4eb2-a18b-fdd73ac23f40 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:28.113153 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6e973-cd8f-4eb2-a18b-fdd73ac23f40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:13:28.118287 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-97c6e973-cd8f-4eb2-a18b-fdd73ac23f40 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:13:28.118287 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:28.118287 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:28.118287 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:28.118287 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:28.118287 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:28.118287 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:28.118287 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:28.118287 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:28.118287 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:28.118287 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:28.118287 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:28.118287 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:13:28.118287 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:13:28.118287 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:13:28.118287 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:13:28.119263 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:13:28.119263 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:13:28.119263 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:13:28.119263 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:13:28.119263 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:28.119263 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:28.119263 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:28.119263 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:28.119263 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:28.119263 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:28.119263 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:28.119263 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:28.119263 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:28.119263 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:28.119263 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:28.119263 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:28.120095 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:28.120095 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:28.120095 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:28.120095 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:28.120095 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:13:28.120095 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:13:28.120095 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:13:28.120095 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:13:28.120095 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:13:28.120095 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:13:28.120095 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:13:28.120095 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:28.124322 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 342/669] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 14:13:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:28.124964 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6e973-cd8f-4eb2-a18b-fdd73ac23f40 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:28.136178 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d530b-c607-4fda-a560-9dd065b3747f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:28.137192 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d530b-c607-4fda-a560-9dd065b3747f None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:28.137476 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d530b-c607-4fda-a560-9dd065b3747f None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:28.137773 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d530b-c607-4fda-a560-9dd065b3747f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:28.145624 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0d530b-c607-4fda-a560-9dd065b3747f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:28.147084 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0d530b-c607-4fda-a560-9dd065b3747f None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:28.160012 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-bf0d530b-c607-4fda-a560-9dd065b3747f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:28.160012 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:28.160012 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:28.160012 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:28.160012 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:28.160012 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:28.160012 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:28.160012 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:28.160012 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:28.160012 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:28.160012 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:28.160012 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:28.160012 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:13:28.160012 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:13:28.160012 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:13:28.160913 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:13:28.160913 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:28.160913 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:28.160913 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:28.160913 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:28.160913 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:28.160913 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:28.160913 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:28.160913 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:28.160913 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:28.160913 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:28.160913 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:28.160913 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:28.160913 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:28.160913 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:28.160913 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:28.160913 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:28.161848 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:13:28.161848 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:13:28.161848 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:13:28.161848 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:13:28.161848 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:13:28.161848 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:13:28.161848 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:28.161848 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:28.165880 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 328/670] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:13:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:28.166732 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d530b-c607-4fda-a560-9dd065b3747f None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:28.233359 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6e973-cd8f-4eb2-a18b-fdd73ac23f40 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:28.235204 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:28.252338 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:28.274291 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e23852-c250-457a-8234-60b1c2bae664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:28.274688 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d530b-c607-4fda-a560-9dd065b3747f None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:28.274801 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e23852-c250-457a-8234-60b1c2bae664 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:28.275622 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e23852-c250-457a-8234-60b1c2bae664 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:28.275889 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e23852-c250-457a-8234-60b1c2bae664 None admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:28.276844 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e23852-c250-457a-8234-60b1c2bae664 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:28.277150 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e23852-c250-457a-8234-60b1c2bae664 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:28.279504 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e23852-c250-457a-8234-60b1c2bae664 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:28.293517 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 343/671] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 14:13:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:28.294317 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e23852-c250-457a-8234-60b1c2bae664 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:28.318310 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ad54b-c6c7-4fd6-8948-749159be5d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:28.318964 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ad54b-c6c7-4fd6-8948-749159be5d4e None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:28.319240 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ad54b-c6c7-4fd6-8948-749159be5d4e None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:28.319506 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ad54b-c6c7-4fd6-8948-749159be5d4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:28.331020 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750ad54b-c6c7-4fd6-8948-749159be5d4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:28.332438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750ad54b-c6c7-4fd6-8948-749159be5d4e None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:28.346697 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-750ad54b-c6c7-4fd6-8948-749159be5d4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:28.346697 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:28.346697 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:28.346697 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:28.346697 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:28.346697 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:28.346697 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:28.346697 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:28.346697 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:28.346697 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:28.346697 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:28.346697 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:28.346697 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:28.346697 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:28.346697 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:28.346697 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:28.347803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:28.347803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:28.347803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:28.347803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:28.347803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:28.347803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:28.347803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:28.347803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:28.347803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:28.347803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:28.347803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:28.347803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:28.347803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:28.347803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:28.347803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:28.347803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:28.347803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:28.348674 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:28.348674 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:28.348674 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:28.348674 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 329/672] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:28.348674 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ad54b-c6c7-4fd6-8948-749159be5d4e None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:28.409446 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e23852-c250-457a-8234-60b1c2bae664 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:28.446385 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c4b73-34c1-4991-aac4-eac0da1b7602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:28.447918 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c4b73-34c1-4991-aac4-eac0da1b7602 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:28.448450 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c4b73-34c1-4991-aac4-eac0da1b7602 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:28.448891 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c4b73-34c1-4991-aac4-eac0da1b7602 None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:28.449699 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c4b73-34c1-4991-aac4-eac0da1b7602 None admin] RBAC: Adding query filter params (name=service) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:28.450155 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c4b73-34c1-4991-aac4-eac0da1b7602 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:28.451840 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c4b73-34c1-4991-aac4-eac0da1b7602 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:28.453825 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ad54b-c6c7-4fd6-8948-749159be5d4e None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:28.460799 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 344/673] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 14:13:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:28.461679 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c4b73-34c1-4991-aac4-eac0da1b7602 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:28.475319 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:28.492637 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af81b962-f377-4527-aa34-5c5db87c7141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:28.493245 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81b962-f377-4527-aa34-5c5db87c7141 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:28.493517 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81b962-f377-4527-aa34-5c5db87c7141 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:28.494312 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81b962-f377-4527-aa34-5c5db87c7141 None admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:28.495030 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af81b962-f377-4527-aa34-5c5db87c7141 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:28.495332 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af81b962-f377-4527-aa34-5c5db87c7141 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:28.496752 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af81b962-f377-4527-aa34-5c5db87c7141 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:28.508385 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:28.510376 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 330/674] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 14:13:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:28.515377 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af81b962-f377-4527-aa34-5c5db87c7141 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:28.581068 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c4b73-34c1-4991-aac4-eac0da1b7602 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:28.614657 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334a719-1c02-4fce-b6ca-23589b8bf104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:28.615310 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334a719-1c02-4fce-b6ca-23589b8bf104 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:28.615585 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334a719-1c02-4fce-b6ca-23589b8bf104 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:28.615856 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334a719-1c02-4fce-b6ca-23589b8bf104 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:28.616572 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c334a719-1c02-4fce-b6ca-23589b8bf104 None admin] RBAC: Adding query filter params (role.id=be958e9954f04268a9f012c775fc17e5, scope.project.id=36a3ae7bab25471fb989136150b6438c, user.id=79eaaae97aac4282a613939e5b768376) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:28.616902 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c334a719-1c02-4fce-b6ca-23589b8bf104 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:28.618302 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c334a719-1c02-4fce-b6ca-23589b8bf104 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:28.624882 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 345/675] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:13:28 2022] GET /identity/v3/role_assignments?user.id=79eaaae97aac4282a613939e5b768376&scope.project.id=36a3ae7bab25471fb989136150b6438c&role.id=be958e9954f04268a9f012c775fc17e5 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:28.625655 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334a719-1c02-4fce-b6ca-23589b8bf104 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:28.626229 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af81b962-f377-4527-aa34-5c5db87c7141 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:28.677523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf97188-2b65-42b2-aeb0-e7b4fa3609ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:28.678138 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf97188-2b65-42b2-aeb0-e7b4fa3609ca None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:28.678414 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf97188-2b65-42b2-aeb0-e7b4fa3609ca None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:28.678894 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf97188-2b65-42b2-aeb0-e7b4fa3609ca None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:28.679969 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf97188-2b65-42b2-aeb0-e7b4fa3609ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:28.680281 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf97188-2b65-42b2-aeb0-e7b4fa3609ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:28.681811 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf97188-2b65-42b2-aeb0-e7b4fa3609ca None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:28.695441 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 331/676] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:28.697377 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf97188-2b65-42b2-aeb0-e7b4fa3609ca None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:28.745192 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334a719-1c02-4fce-b6ca-23589b8bf104 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:28.747798 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:28.749748 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:28.784896 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbfb283-e17e-4802-8bc8-3086d5740f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:28.785539 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfb283-e17e-4802-8bc8-3086d5740f0e None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:28.785813 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfb283-e17e-4802-8bc8-3086d5740f0e None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:28.786078 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfb283-e17e-4802-8bc8-3086d5740f0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:28.797951 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbfb283-e17e-4802-8bc8-3086d5740f0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:28.799451 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbfb283-e17e-4802-8bc8-3086d5740f0e None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:28.815995 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-cbbfb283-e17e-4802-8bc8-3086d5740f0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:28.815995 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:28.815995 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:28.815995 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:28.815995 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:28.815995 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:28.815995 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:28.815995 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:28.815995 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:28.815995 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:28.815995 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:28.815995 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:28.815995 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:28.815995 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:28.815995 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:28.815995 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:28.816979 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:28.816979 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:28.816979 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:28.816979 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:28.816979 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:28.816979 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:28.816979 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:28.816979 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:28.816979 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:28.816979 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:28.816979 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:28.816979 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:28.816979 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:28.816979 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:28.816979 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:28.816979 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:28.816979 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:28.817876 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:28.817876 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:28.817876 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:28.817876 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 346/677] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:28.819500 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d6cbcc-db71-49b7-abaf-e721a0354860 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:28.819696 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d6cbcc-db71-49b7-abaf-e721a0354860 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:28.819942 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d6cbcc-db71-49b7-abaf-e721a0354860 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:28.825602 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 347/678] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:28.825602 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d6cbcc-db71-49b7-abaf-e721a0354860 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:28.827306 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf97188-2b65-42b2-aeb0-e7b4fa3609ca None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:28.875618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff256a8-ca69-4278-aee6-a37d8fb3c7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:28.876531 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff256a8-ca69-4278-aee6-a37d8fb3c7c2 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:28.876963 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff256a8-ca69-4278-aee6-a37d8fb3c7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:28.877232 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff256a8-ca69-4278-aee6-a37d8fb3c7c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:28.878578 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff256a8-ca69-4278-aee6-a37d8fb3c7c2 None admin] RBAC: Adding query filter params (role.id=be958e9954f04268a9f012c775fc17e5, scope.project.id=36a3ae7bab25471fb989136150b6438c, user.id=63921623056045a9b97b2de3ca418138) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:28.878578 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff256a8-ca69-4278-aee6-a37d8fb3c7c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:28.879825 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff256a8-ca69-4278-aee6-a37d8fb3c7c2 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:28.888933 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 332/679] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:13:28 2022] GET /identity/v3/role_assignments?user.id=63921623056045a9b97b2de3ca418138&scope.project.id=36a3ae7bab25471fb989136150b6438c&role.id=be958e9954f04268a9f012c775fc17e5 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:28.889758 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff256a8-ca69-4278-aee6-a37d8fb3c7c2 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:28.960335 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d6cbcc-db71-49b7-abaf-e721a0354860 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:29.013679 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f995d09-da34-4341-862a-61ac1d31749a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:29.014679 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f995d09-da34-4341-862a-61ac1d31749a None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:29.015005 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f995d09-da34-4341-862a-61ac1d31749a None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:29.015732 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f995d09-da34-4341-862a-61ac1d31749a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:29.018377 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff256a8-ca69-4278-aee6-a37d8fb3c7c2 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:29.020252 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:29.027403 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f995d09-da34-4341-862a-61ac1d31749a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:29.030334 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f995d09-da34-4341-862a-61ac1d31749a None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:29.047794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-9f995d09-da34-4341-862a-61ac1d31749a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:29.047794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:29.047794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:29.047794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:29.047794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:29.047794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:29.047794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:29.047794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:29.047794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:29.047794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:29.047794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:29.047794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:29.047794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:13:29.047794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:13:29.047794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:13:29.048969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:13:29.048969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:29.048969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:29.048969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:29.048969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:29.048969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:29.048969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:29.048969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:29.048969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:29.048969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:29.048969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:29.048969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:29.048969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:29.048969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:29.048969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:29.048969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:29.048969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:29.050068 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:13:29.050068 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:13:29.050068 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:13:29.050068 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:13:29.050068 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:13:29.050068 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:13:29.050068 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:29.050068 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:29.050068 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 348/680] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:13:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:29.053967 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50bf79-259f-44a8-83dd-09b43db0c225 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:29.053967 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50bf79-259f-44a8-83dd-09b43db0c225 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:29.053967 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50bf79-259f-44a8-83dd-09b43db0c225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:29.063063 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8057fbe-600c-46bb-95a2-2034e2828683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:29.063834 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8057fbe-600c-46bb-95a2-2034e2828683 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:29.064227 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8057fbe-600c-46bb-95a2-2034e2828683 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:29.064593 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8057fbe-600c-46bb-95a2-2034e2828683 None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:29.065412 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8057fbe-600c-46bb-95a2-2034e2828683 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:29.065810 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8057fbe-600c-46bb-95a2-2034e2828683 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:29.067388 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8057fbe-600c-46bb-95a2-2034e2828683 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:29.076725 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-6d50bf79-259f-44a8-83dd-09b43db0c225 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:29.077953 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-6d50bf79-259f-44a8-83dd-09b43db0c225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:29.082966 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 333/681] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:29.084540 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8057fbe-600c-46bb-95a2-2034e2828683 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:29.092459 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:29.159575 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 349/682] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:13:29.160650 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50bf79-259f-44a8-83dd-09b43db0c225 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:29.222068 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8057fbe-600c-46bb-95a2-2034e2828683 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:29.281920 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202409e-c220-410f-8e1f-04152bb602be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:29.282539 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202409e-c220-410f-8e1f-04152bb602be None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:29.283099 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202409e-c220-410f-8e1f-04152bb602be None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:29.283374 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202409e-c220-410f-8e1f-04152bb602be None admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:29.284073 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2202409e-c220-410f-8e1f-04152bb602be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:29.284379 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2202409e-c220-410f-8e1f-04152bb602be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:29.285819 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2202409e-c220-410f-8e1f-04152bb602be None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:29.317805 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50bf79-259f-44a8-83dd-09b43db0c225 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:29.320311 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 334/683] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 14:13:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:29.325874 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202409e-c220-410f-8e1f-04152bb602be None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:29.372539 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:29.395004 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a7d35-dd08-47aa-aded-1b7d7d4d907a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:29.395730 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a7d35-dd08-47aa-aded-1b7d7d4d907a None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:29.395929 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a7d35-dd08-47aa-aded-1b7d7d4d907a None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:29.396494 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a7d35-dd08-47aa-aded-1b7d7d4d907a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:29.414464 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191a7d35-dd08-47aa-aded-1b7d7d4d907a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:29.420344 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191a7d35-dd08-47aa-aded-1b7d7d4d907a None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:29.442690 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-191a7d35-dd08-47aa-aded-1b7d7d4d907a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 24 14:13:29.442690 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:29.442690 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:29.442690 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:29.442690 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:29.442690 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:29.442690 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:29.442690 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:29.442690 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:29.442690 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:29.442690 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:29.442690 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:29.442690 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:13:29.442690 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:13:29.442690 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:13:29.442690 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:13:29.443681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:29.443681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:29.443681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:13:29.443681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:29.443681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:13:29.443681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:29.443681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:29.443681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:29.443681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:29.443681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:29.443681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:29.443681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:29.443681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:29.443681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:29.443681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:29.443681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:29.443681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:29.443681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:29.444607 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:29.444607 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:29.444607 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:13:29.444607 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:13:29.444607 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:13:29.444607 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:13:29.444607 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:13:29.444607 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:13:29.444607 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 24 14:13:29.444607 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:29.449778 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 350/684] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 14:13:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:29.454671 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a7d35-dd08-47aa-aded-1b7d7d4d907a None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:29.467197 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:29.501330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202409e-c220-410f-8e1f-04152bb602be None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:29.540938 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30cea9-5b62-4e11-9a26-e0516ec5f1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:29.541906 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30cea9-5b62-4e11-9a26-e0516ec5f1dc None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:29.542415 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30cea9-5b62-4e11-9a26-e0516ec5f1dc None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:29.542988 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30cea9-5b62-4e11-9a26-e0516ec5f1dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:29.544032 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb30cea9-5b62-4e11-9a26-e0516ec5f1dc None admin] RBAC: Adding query filter params (role.id=be958e9954f04268a9f012c775fc17e5, scope.project.id=36a3ae7bab25471fb989136150b6438c, user.id=a9348164132846d2b593ddfa29a820eb) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:29.544537 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb30cea9-5b62-4e11-9a26-e0516ec5f1dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:29.546236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb30cea9-5b62-4e11-9a26-e0516ec5f1dc None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:29.559278 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 335/685] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:13:29 2022] GET /identity/v3/role_assignments?user.id=a9348164132846d2b593ddfa29a820eb&scope.project.id=36a3ae7bab25471fb989136150b6438c&role.id=be958e9954f04268a9f012c775fc17e5 => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:13:29.560693 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30cea9-5b62-4e11-9a26-e0516ec5f1dc None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:29.583084 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a7d35-dd08-47aa-aded-1b7d7d4d907a None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:29.626691 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5e1c29-ca9a-4d68-9a30-b1e21ed28f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:29.627286 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e1c29-ca9a-4d68-9a30-b1e21ed28f4d None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:29.627569 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e1c29-ca9a-4d68-9a30-b1e21ed28f4d None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:29.627845 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e1c29-ca9a-4d68-9a30-b1e21ed28f4d None admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:29.628588 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5e1c29-ca9a-4d68-9a30-b1e21ed28f4d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:29.628895 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5e1c29-ca9a-4d68-9a30-b1e21ed28f4d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:29.632174 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5e1c29-ca9a-4d68-9a30-b1e21ed28f4d None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:29.654349 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 351/686] 10.222.0.52 () {56 vars in 1135 bytes} [Sat Sep 24 14:13:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:13:29.664636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5e1c29-ca9a-4d68-9a30-b1e21ed28f4d None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:29.751632 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:29.785135 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30cea9-5b62-4e11-9a26-e0516ec5f1dc None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:29.811270 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:29.853862 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9978759-2cad-4f4b-8623-a5a6210aabd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:29.855506 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9978759-2cad-4f4b-8623-a5a6210aabd0 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:29.855974 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9978759-2cad-4f4b-8623-a5a6210aabd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:29.856426 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9978759-2cad-4f4b-8623-a5a6210aabd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:29.860569 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5e1c29-ca9a-4d68-9a30-b1e21ed28f4d None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:29.870503 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9978759-2cad-4f4b-8623-a5a6210aabd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:29.872799 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9978759-2cad-4f4b-8623-a5a6210aabd0 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:29.891574 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-e9978759-2cad-4f4b-8623-a5a6210aabd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:29.891574 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:29.891574 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:29.891574 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:29.891574 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:29.891574 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:29.891574 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:29.891574 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:29.891574 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:29.891574 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:29.891574 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:29.891574 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:29.891574 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:29.891574 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:29.891574 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:29.891574 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:29.892542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:29.892542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:29.892542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:29.892542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:29.892542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:29.892542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:29.892542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:29.892542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:29.892542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:29.892542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:29.892542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:29.892542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:29.892542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:29.892542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:29.892542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:29.892542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:29.892542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:29.893427 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:29.893427 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:29.893427 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:29.896101 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 336/687] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:29.902450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9978759-2cad-4f4b-8623-a5a6210aabd0 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:29.906056 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e39349-73ff-47ea-91fb-9593376c9a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:29.907078 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e39349-73ff-47ea-91fb-9593376c9a84 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:29.907509 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e39349-73ff-47ea-91fb-9593376c9a84 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:29.909075 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e39349-73ff-47ea-91fb-9593376c9a84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:30.002160 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e39349-73ff-47ea-91fb-9593376c9a84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:30.003971 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e39349-73ff-47ea-91fb-9593376c9a84 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:30.023612 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9978759-2cad-4f4b-8623-a5a6210aabd0 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:30.061928 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f3af6a-196e-4f24-b90c-4ea83ea4b8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:30.061928 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f3af6a-196e-4f24-b90c-4ea83ea4b8bf None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:30.061928 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f3af6a-196e-4f24-b90c-4ea83ea4b8bf None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:30.061928 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f3af6a-196e-4f24-b90c-4ea83ea4b8bf None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:30.062376 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f3af6a-196e-4f24-b90c-4ea83ea4b8bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:30.062376 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f3af6a-196e-4f24-b90c-4ea83ea4b8bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:30.063743 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f3af6a-196e-4f24-b90c-4ea83ea4b8bf None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:30.074669 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:30.074879 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:30.078982 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 337/688] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:13:30.084045 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f3af6a-196e-4f24-b90c-4ea83ea4b8bf None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:30.117294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 352/689] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:13:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:13:30.122941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e39349-73ff-47ea-91fb-9593376c9a84 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:30.193524 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f3af6a-196e-4f24-b90c-4ea83ea4b8bf None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:30.234252 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd97b8f-fe11-4a74-83bd-7823d5b49a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:30.235685 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd97b8f-fe11-4a74-83bd-7823d5b49a92 None admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:30.236144 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd97b8f-fe11-4a74-83bd-7823d5b49a92 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:30.236611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd97b8f-fe11-4a74-83bd-7823d5b49a92 None admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:30.237427 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd97b8f-fe11-4a74-83bd-7823d5b49a92 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:30.239042 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd97b8f-fe11-4a74-83bd-7823d5b49a92 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:30.268715 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 338/690] 10.222.0.52 () {60 vars in 1132 bytes} [Sat Sep 24 14:13:30 2022] POST /identity/v3/users => generated 254 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:13:30.322536 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e39349-73ff-47ea-91fb-9593376c9a84 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:30.360935 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2caa2-158a-41ae-9dfa-8618721c013f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:30.361886 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2caa2-158a-41ae-9dfa-8618721c013f None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:30.362369 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2caa2-158a-41ae-9dfa-8618721c013f None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:30.362842 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2caa2-158a-41ae-9dfa-8618721c013f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:30.375464 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e2caa2-158a-41ae-9dfa-8618721c013f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:30.377269 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e2caa2-158a-41ae-9dfa-8618721c013f None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:30.394891 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-11e2caa2-158a-41ae-9dfa-8618721c013f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:30.394891 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:30.394891 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:30.394891 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:30.394891 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:30.394891 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:30.394891 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:30.394891 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:30.394891 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:30.394891 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:30.394891 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:30.394891 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:30.394891 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:30.394891 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:30.394891 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:30.394891 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:30.395926 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:30.395926 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:30.395926 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:30.395926 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:30.395926 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:30.395926 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:30.395926 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:30.395926 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:30.395926 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:30.395926 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:30.395926 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:30.395926 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:30.395926 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:30.395926 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:30.395926 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:30.395926 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:30.395926 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:30.396800 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:30.396800 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:30.396800 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:30.398707 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 353/691] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:13:30.407980 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd97b8f-fe11-4a74-83bd-7823d5b49a92 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:30.477334 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:30.535371 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd97b8f-fe11-4a74-83bd-7823d5b49a92 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:30.569772 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85850bb2-3202-40a4-8fbe-d7d2c8308cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:30.570387 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85850bb2-3202-40a4-8fbe-d7d2c8308cbe None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:30.570664 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85850bb2-3202-40a4-8fbe-d7d2c8308cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:30.570965 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85850bb2-3202-40a4-8fbe-d7d2c8308cbe None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:30.571615 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85850bb2-3202-40a4-8fbe-d7d2c8308cbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:30.572471 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85850bb2-3202-40a4-8fbe-d7d2c8308cbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:30.573826 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85850bb2-3202-40a4-8fbe-d7d2c8308cbe None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:30.587951 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 339/692] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:30.593461 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2caa2-158a-41ae-9dfa-8618721c013f None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:30.613716 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:30.700343 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2caa2-158a-41ae-9dfa-8618721c013f None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:30.739423 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32403810-ec40-4f34-96cc-49529d02d02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:30.739982 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32403810-ec40-4f34-96cc-49529d02d02f None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:30.740263 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32403810-ec40-4f34-96cc-49529d02d02f None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:30.740531 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32403810-ec40-4f34-96cc-49529d02d02f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:30.747772 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32403810-ec40-4f34-96cc-49529d02d02f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:30.749218 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32403810-ec40-4f34-96cc-49529d02d02f None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:30.760943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-32403810-ec40-4f34-96cc-49529d02d02f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:30.760943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:30.760943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:30.760943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:30.760943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:30.760943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:30.760943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:30.760943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:30.760943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:30.760943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:30.760943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:30.760943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:30.760943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:13:30.760943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:13:30.760943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:13:30.761853 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:13:30.761853 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:30.761853 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:30.761853 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:30.761853 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:30.761853 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:30.761853 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:30.761853 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:30.761853 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:30.761853 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:30.761853 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:30.761853 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:30.761853 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:30.761853 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:30.761853 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:30.761853 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:30.761853 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:30.762747 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:13:30.762747 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:13:30.762747 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:13:30.762747 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:13:30.762747 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:13:30.762747 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:13:30.762747 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:30.762747 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:30.762747 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 354/693] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:13:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:30.768244 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85850bb2-3202-40a4-8fbe-d7d2c8308cbe None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:30.882340 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85850bb2-3202-40a4-8fbe-d7d2c8308cbe None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:30.903049 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:30.928783 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223e8b79-04fb-4873-9599-f684ce41b739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:30.929406 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e8b79-04fb-4873-9599-f684ce41b739 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:30.929684 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e8b79-04fb-4873-9599-f684ce41b739 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:30.929950 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e8b79-04fb-4873-9599-f684ce41b739 None admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:30.930643 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223e8b79-04fb-4873-9599-f684ce41b739 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:30.930976 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223e8b79-04fb-4873-9599-f684ce41b739 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:30.932967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223e8b79-04fb-4873-9599-f684ce41b739 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:30.948955 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 340/694] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 14:13:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:30.955246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32403810-ec40-4f34-96cc-49529d02d02f None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:31.063899 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:31.100728 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32403810-ec40-4f34-96cc-49529d02d02f None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:31.145019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb371c-f1fc-4642-9d2c-27262ea31fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:31.146027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb371c-f1fc-4642-9d2c-27262ea31fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:31.146512 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb371c-f1fc-4642-9d2c-27262ea31fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:31.147015 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb371c-f1fc-4642-9d2c-27262ea31fe0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:31.147940 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb371c-f1fc-4642-9d2c-27262ea31fe0 None admin] RBAC: Adding query filter params (role.id=be958e9954f04268a9f012c775fc17e5, scope.project.id=36a3ae7bab25471fb989136150b6438c, user.id=879f296733114f6f844a98aaefe35e92) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:31.148465 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb371c-f1fc-4642-9d2c-27262ea31fe0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:31.150163 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb371c-f1fc-4642-9d2c-27262ea31fe0 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:31.156664 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 355/695] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:13:30 2022] GET /identity/v3/role_assignments?user.id=879f296733114f6f844a98aaefe35e92&scope.project.id=36a3ae7bab25471fb989136150b6438c&role.id=be958e9954f04268a9f012c775fc17e5 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:31.231286 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369894e-dca3-41e8-baa1-840822db4d88 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:31.231631 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369894e-dca3-41e8-baa1-840822db4d88 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:31.231846 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369894e-dca3-41e8-baa1-840822db4d88 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:31.233525 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 341/696] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:31.238835 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b29e6-5e06-4c2c-9d43-e6a1886dc48c None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:31.239395 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b29e6-5e06-4c2c-9d43-e6a1886dc48c None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:31.239890 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b29e6-5e06-4c2c-9d43-e6a1886dc48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:31.264556 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-0a3b29e6-5e06-4c2c-9d43-e6a1886dc48c None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:31.265776 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-0a3b29e6-5e06-4c2c-9d43-e6a1886dc48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:31.377093 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 356/697] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:13:31.647133 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f64f74b-35ad-4f5a-b79c-a9768bab9499 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:31.647639 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f64f74b-35ad-4f5a-b79c-a9768bab9499 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:31.647719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f64f74b-35ad-4f5a-b79c-a9768bab9499 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:31.649144 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 342/698] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:31.655323 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d7db0-fe59-4978-ab9b-c87d57a6ca18 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:31.655615 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d7db0-fe59-4978-ab9b-c87d57a6ca18 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:31.655892 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d7db0-fe59-4978-ab9b-c87d57a6ca18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:31.677742 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-827d7db0-fe59-4978-ab9b-c87d57a6ca18 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:31.678066 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f64f74b-35ad-4f5a-b79c-a9768bab9499 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:31.679028 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-827d7db0-fe59-4978-ab9b-c87d57a6ca18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:31.801557 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:31.823135 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 357/699] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:13:31.860464 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f64f74b-35ad-4f5a-b79c-a9768bab9499 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:31.908113 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29642efa-b597-413f-bb50-19ae7bfce20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:31.908677 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29642efa-b597-413f-bb50-19ae7bfce20f None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:31.908958 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29642efa-b597-413f-bb50-19ae7bfce20f None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:31.909233 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29642efa-b597-413f-bb50-19ae7bfce20f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:31.928396 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29642efa-b597-413f-bb50-19ae7bfce20f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:13:31.934335 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-29642efa-b597-413f-bb50-19ae7bfce20f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:13:31.934335 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:31.934335 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:31.934335 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:31.934335 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:31.934335 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:31.934335 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:31.934335 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:31.934335 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:31.934335 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:31.934335 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:31.934335 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:31.934335 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:13:31.934335 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:13:31.934335 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:13:31.934335 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:13:31.935262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:13:31.935262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:13:31.935262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:13:31.935262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:13:31.935262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:31.935262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:31.935262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:31.935262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:31.935262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:31.935262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:31.935262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:31.935262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:31.935262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:31.935262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:31.935262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:31.935262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:31.936107 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:31.936107 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:31.936107 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:31.936107 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:31.936107 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:13:31.936107 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:13:31.936107 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:13:31.936107 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:13:31.936107 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:13:31.936107 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:13:31.936107 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:13:31.936107 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:31.936107 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 343/700] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 14:13:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:13:31.941214 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d7db0-fe59-4978-ab9b-c87d57a6ca18 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:32.091715 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d7db0-fe59-4978-ab9b-c87d57a6ca18 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:32.130870 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e86caf-b70d-472c-959c-d8460bddb0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:32.131496 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e86caf-b70d-472c-959c-d8460bddb0e8 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:32.131823 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e86caf-b70d-472c-959c-d8460bddb0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:32.132098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e86caf-b70d-472c-959c-d8460bddb0e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:32.132761 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e86caf-b70d-472c-959c-d8460bddb0e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:32.133063 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e86caf-b70d-472c-959c-d8460bddb0e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:32.134491 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e86caf-b70d-472c-959c-d8460bddb0e8 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:32.145949 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 358/701] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:13:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:32.152031 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29642efa-b597-413f-bb50-19ae7bfce20f None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:32.198938 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:32.283790 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29642efa-b597-413f-bb50-19ae7bfce20f None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:32.305782 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e86caf-b70d-472c-959c-d8460bddb0e8 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:32.325632 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac5e3d-fd27-41e0-b6f9-8d748188f5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:32.326282 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac5e3d-fd27-41e0-b6f9-8d748188f5b2 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:32.326565 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac5e3d-fd27-41e0-b6f9-8d748188f5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:32.326854 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac5e3d-fd27-41e0-b6f9-8d748188f5b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:32.348815 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ac5e3d-fd27-41e0-b6f9-8d748188f5b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:32.350271 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ac5e3d-fd27-41e0-b6f9-8d748188f5b2 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:32.375258 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-30ac5e3d-fd27-41e0-b6f9-8d748188f5b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:32.375258 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:32.375258 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:32.375258 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:32.375258 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:32.375258 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:32.375258 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:32.375258 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:32.375258 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:32.375258 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:32.375258 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:32.375258 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:32.375258 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:32.375258 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:32.375258 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:32.375258 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:32.376260 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:32.376260 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:32.376260 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:32.376260 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:32.376260 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:32.376260 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:32.376260 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:32.376260 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:32.376260 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:32.376260 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:32.376260 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:32.376260 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:32.376260 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:32.376260 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:32.376260 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:32.376260 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:32.376260 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:32.377139 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:32.377139 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:32.377139 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:32.377139 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 344/702] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:32.389636 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac5e3d-fd27-41e0-b6f9-8d748188f5b2 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:32.430997 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:32.470264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e86caf-b70d-472c-959c-d8460bddb0e8 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:32.509242 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dafd8c-7c6b-4e10-8339-5ab81a7c95d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:32.510333 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dafd8c-7c6b-4e10-8339-5ab81a7c95d4 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:32.510849 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dafd8c-7c6b-4e10-8339-5ab81a7c95d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:32.511363 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dafd8c-7c6b-4e10-8339-5ab81a7c95d4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:32.512147 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dafd8c-7c6b-4e10-8339-5ab81a7c95d4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:32.513726 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dafd8c-7c6b-4e10-8339-5ab81a7c95d4 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:32.517630 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:32.523367 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac5e3d-fd27-41e0-b6f9-8d748188f5b2 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:32.528414 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-d9dafd8c-7c6b-4e10-8339-5ab81a7c95d4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 14:13:32.528414 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:32.528414 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:13:32.528414 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:13:32.528414 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:13:32.528414 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:13:32.528414 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:13:32.528414 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:13:32.528414 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 14:13:32.528414 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:32.528414 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:13:32.528414 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:32.528414 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:32.528414 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:32.528414 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:32.528414 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:32.528414 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:32.529452 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:32.529452 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:32.529452 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:32.529452 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:32.529452 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:32.529452 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:32.529452 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:13:32.529452 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:13:32.529452 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:13:32.529452 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:13:32.529452 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:32.529452 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:32.529452 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:32.529452 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:32.529452 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:32.529452 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:32.530430 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:32.530430 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:32.530430 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:32.530430 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:32.530430 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:32.530430 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:32.530430 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:32.530430 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:32.530430 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:32.530430 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:32.530430 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:13:32.530430 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:13:32.530430 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 14:13:32.530430 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:32.537429 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 359/703] 10.222.0.52 () {56 vars in 1129 bytes} [Sat Sep 24 14:13:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:13:32.537429 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ce864-1b9f-42d1-bb86-bcd978860719 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:32.537429 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ce864-1b9f-42d1-bb86-bcd978860719 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:32.537429 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ce864-1b9f-42d1-bb86-bcd978860719 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:32.537767 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 360/704] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:32.539466 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199ce864-1b9f-42d1-bb86-bcd978860719 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:32.578038 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02405c56-f2b8-4397-8db8-296882ee4de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:32.579045 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02405c56-f2b8-4397-8db8-296882ee4de4 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:32.579527 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02405c56-f2b8-4397-8db8-296882ee4de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:32.580259 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02405c56-f2b8-4397-8db8-296882ee4de4 None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:32.581978 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02405c56-f2b8-4397-8db8-296882ee4de4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:32.582509 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02405c56-f2b8-4397-8db8-296882ee4de4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:32.584124 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02405c56-f2b8-4397-8db8-296882ee4de4 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:32.600571 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 345/705] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:13:32.602277 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e797722-5d67-44f4-b0bb-2bbe4d452819 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:32.602784 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e797722-5d67-44f4-b0bb-2bbe4d452819 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:32.603810 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e797722-5d67-44f4-b0bb-2bbe4d452819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:32.627335 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-6e797722-5d67-44f4-b0bb-2bbe4d452819 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:32.629356 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-6e797722-5d67-44f4-b0bb-2bbe4d452819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:32.655389 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199ce864-1b9f-42d1-bb86-bcd978860719 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:32.684211 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:32.694468 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578a4c6-06af-4b27-b7df-1953ef02f5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:32.695035 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578a4c6-06af-4b27-b7df-1953ef02f5f3 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:32.695321 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578a4c6-06af-4b27-b7df-1953ef02f5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:32.695634 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578a4c6-06af-4b27-b7df-1953ef02f5f3 None admin] PATH_INFO: `/v3/services` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:32.696273 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f578a4c6-06af-4b27-b7df-1953ef02f5f3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:32.696577 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f578a4c6-06af-4b27-b7df-1953ef02f5f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:32.698585 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f578a4c6-06af-4b27-b7df-1953ef02f5f3 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:32.707970 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 361/706] 10.222.0.52 () {56 vars in 1124 bytes} [Sat Sep 24 14:13:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:32.707970 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578a4c6-06af-4b27-b7df-1953ef02f5f3 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:32.734476 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 346/707] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:13:32.734476 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e797722-5d67-44f4-b0bb-2bbe4d452819 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:32.852271 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:32.873328 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e797722-5d67-44f4-b0bb-2bbe4d452819 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:32.893750 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578a4c6-06af-4b27-b7df-1953ef02f5f3 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:32.914097 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bc7fb2-1812-4beb-880d-71ece19bb6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:32.915132 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc7fb2-1812-4beb-880d-71ece19bb6ca None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:32.915666 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc7fb2-1812-4beb-880d-71ece19bb6ca None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:32.916140 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc7fb2-1812-4beb-880d-71ece19bb6ca None admin] PATH_INFO: `/v3/services` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:32.916971 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bc7fb2-1812-4beb-880d-71ece19bb6ca None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:32.917548 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bc7fb2-1812-4beb-880d-71ece19bb6ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:32.921527 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bc7fb2-1812-4beb-880d-71ece19bb6ca None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:32.929676 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 347/708] 10.222.0.52 () {56 vars in 1124 bytes} [Sat Sep 24 14:13:32 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:32.963881 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7efe23-4c3e-458b-95a7-2158792e9dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:32.964538 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7efe23-4c3e-458b-95a7-2158792e9dd4 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:32.964818 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7efe23-4c3e-458b-95a7-2158792e9dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:32.965088 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7efe23-4c3e-458b-95a7-2158792e9dd4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:32.976362 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7efe23-4c3e-458b-95a7-2158792e9dd4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:32.977837 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7efe23-4c3e-458b-95a7-2158792e9dd4 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:32.999032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-6a7efe23-4c3e-458b-95a7-2158792e9dd4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 14:13:32.999032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:32.999032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:32.999032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:32.999032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:32.999032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:32.999032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:32.999032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:32.999032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:32.999032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:32.999032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:32.999032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:32.999032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:13:32.999032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:13:32.999032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:13:32.999032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:13:33.000008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:33.000008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:33.000008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:13:33.000008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:33.000008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:13:33.000008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:33.000008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:33.000008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:33.000008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:33.000008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:33.000008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:33.000008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:33.000008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:33.000008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:33.000008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:33.000008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:33.000008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:33.000008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:33.000930 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:33.000930 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:33.000930 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:13:33.000930 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:13:33.000930 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:13:33.000930 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:13:33.000930 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:13:33.000930 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:13:33.000930 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 14:13:33.000930 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:33.000930 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 362/709] 10.222.0.52 () {56 vars in 1093 bytes} [Sat Sep 24 14:13:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:33.005927 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bc7fb2-1812-4beb-880d-71ece19bb6ca None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:33.130959 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bc7fb2-1812-4beb-880d-71ece19bb6ca None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:33.169467 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f48533-a5f1-4615-8747-dedbdf4a7329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:33.170440 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f48533-a5f1-4615-8747-dedbdf4a7329 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:33.170981 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f48533-a5f1-4615-8747-dedbdf4a7329 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:33.171473 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f48533-a5f1-4615-8747-dedbdf4a7329 None admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:33.172412 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f48533-a5f1-4615-8747-dedbdf4a7329 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:33.172932 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f48533-a5f1-4615-8747-dedbdf4a7329 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:33.174577 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f48533-a5f1-4615-8747-dedbdf4a7329 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:33.185490 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:33.195017 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 348/710] 10.222.0.52 () {56 vars in 1133 bytes} [Sat Sep 24 14:13:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:33.196209 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a718d1f-a488-4107-8d41-8ee1d9a1677d None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:33.197170 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a718d1f-a488-4107-8d41-8ee1d9a1677d None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:33.197269 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a718d1f-a488-4107-8d41-8ee1d9a1677d None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:33.198581 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 363/711] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:33.204014 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a718d1f-a488-4107-8d41-8ee1d9a1677d None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:33.204139 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93edeee0-9b77-496b-8361-dff100dfa488 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:33.204310 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93edeee0-9b77-496b-8361-dff100dfa488 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:33.204568 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93edeee0-9b77-496b-8361-dff100dfa488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:33.223660 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-93edeee0-9b77-496b-8361-dff100dfa488 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:33.224847 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-93edeee0-9b77-496b-8361-dff100dfa488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:33.293057 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:33.306392 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 349/712] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:13:33.307286 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93edeee0-9b77-496b-8361-dff100dfa488 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:33.384025 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a718d1f-a488-4107-8d41-8ee1d9a1677d None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:33.452843 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa9d8b-64ca-4929-a500-b6f7962d7646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:33.453848 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa9d8b-64ca-4929-a500-b6f7962d7646 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:33.454347 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa9d8b-64ca-4929-a500-b6f7962d7646 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:33.454838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa9d8b-64ca-4929-a500-b6f7962d7646 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:33.501555 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93edeee0-9b77-496b-8361-dff100dfa488 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:33.566541 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:33.582157 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a46506c-e372-4cd9-a144-db66cd29a560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:33.583280 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46506c-e372-4cd9-a144-db66cd29a560 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:33.583781 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46506c-e372-4cd9-a144-db66cd29a560 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:33.584303 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46506c-e372-4cd9-a144-db66cd29a560 None admin] PATH_INFO: `/v3/services/network` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:33.585089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a46506c-e372-4cd9-a144-db66cd29a560 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:33.586828 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a46506c-e372-4cd9-a144-db66cd29a560 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:33.598667 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa9d8b-64ca-4929-a500-b6f7962d7646 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:33.599608 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-1a46506c-e372-4cd9-a144-db66cd29a560 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 14:13:33.599608 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:33.599608 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:13:33.599608 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:13:33.599608 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:13:33.599608 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:13:33.599608 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:13:33.599608 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:13:33.599608 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 14:13:33.599608 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:33.599608 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:13:33.599608 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:33.599608 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:33.599608 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:33.599608 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:33.599608 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:33.599608 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:33.600610 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:33.600610 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:33.600610 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:33.600610 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:33.600610 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:33.600610 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:33.600610 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:13:33.600610 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:13:33.600610 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:13:33.600610 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:13:33.600610 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:33.600610 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:33.600610 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:33.600610 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:33.600610 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:33.600610 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:33.601430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:33.601430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:33.601430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:33.601430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:33.601430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:33.601430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:33.601430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:33.601430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:33.601430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:33.601430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:33.601430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:13:33.601430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:13:33.601430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 14:13:33.601430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:33.602156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa9d8b-64ca-4929-a500-b6f7962d7646 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:33.606087 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 350/713] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:13:33 2022] GET /identity/v3/services/network => generated 88 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:33.607758 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a46506c-e372-4cd9-a144-db66cd29a560 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:33.659168 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:33.715786 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 364/714] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:13:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:13:33.717068 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa9d8b-64ca-4929-a500-b6f7962d7646 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:33.730289 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a46506c-e372-4cd9-a144-db66cd29a560 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:33.768554 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2416304-0678-4c27-96f3-138cf1ccfe54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:33.769174 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2416304-0678-4c27-96f3-138cf1ccfe54 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:33.769458 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2416304-0678-4c27-96f3-138cf1ccfe54 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:33.769735 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2416304-0678-4c27-96f3-138cf1ccfe54 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:33.780991 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2416304-0678-4c27-96f3-138cf1ccfe54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:13:33.787236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-c2416304-0678-4c27-96f3-138cf1ccfe54 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:13:33.787236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:33.787236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:33.787236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:33.787236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:33.787236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:33.787236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:33.787236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:33.787236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:33.787236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:33.787236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:33.787236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:33.787236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:13:33.787236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:13:33.787236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:13:33.787236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:13:33.788239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:13:33.788239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:13:33.788239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:13:33.788239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:13:33.788239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:33.788239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:33.788239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:33.788239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:33.788239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:33.788239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:33.788239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:33.788239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:33.788239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:33.788239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:33.788239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:33.788239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:33.789063 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:33.789063 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:33.789063 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:33.789063 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:33.789063 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:13:33.789063 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:13:33.789063 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:13:33.789063 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:13:33.789063 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:13:33.789063 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:13:33.789063 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:13:33.789063 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:33.789063 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 351/715] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 14:13:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:33.789739 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2416304-0678-4c27-96f3-138cf1ccfe54 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:33.829677 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa9d8b-64ca-4929-a500-b6f7962d7646 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:33.846171 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:33.871662 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219964b5-5c4c-4f95-a033-98759e305a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:33.872288 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219964b5-5c4c-4f95-a033-98759e305a34 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:33.872962 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219964b5-5c4c-4f95-a033-98759e305a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:33.873234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219964b5-5c4c-4f95-a033-98759e305a34 None admin] PATH_INFO: `/v3/services` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:33.873863 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219964b5-5c4c-4f95-a033-98759e305a34 None admin] RBAC: Adding query filter params (name=network) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:33.874163 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219964b5-5c4c-4f95-a033-98759e305a34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:33.875932 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219964b5-5c4c-4f95-a033-98759e305a34 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:33.881246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 365/716] 10.222.0.52 () {56 vars in 1112 bytes} [Sat Sep 24 14:13:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:33.882545 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219964b5-5c4c-4f95-a033-98759e305a34 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:33.900688 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2416304-0678-4c27-96f3-138cf1ccfe54 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:33.905434 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:33.937262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5b0adb-6ff1-418e-9e40-aa5295c88b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:33.938179 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b0adb-6ff1-418e-9e40-aa5295c88b37 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:33.938608 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b0adb-6ff1-418e-9e40-aa5295c88b37 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:33.938906 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b0adb-6ff1-418e-9e40-aa5295c88b37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:33.950657 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5b0adb-6ff1-418e-9e40-aa5295c88b37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:33.952133 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5b0adb-6ff1-418e-9e40-aa5295c88b37 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:33.966890 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-8b5b0adb-6ff1-418e-9e40-aa5295c88b37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:33.966890 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:33.966890 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:33.966890 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:33.966890 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:33.966890 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:33.966890 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:33.966890 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:33.966890 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:33.966890 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:33.966890 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:33.966890 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:33.966890 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:33.966890 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:33.966890 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:33.966890 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:33.967852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:33.967852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:33.967852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:33.967852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:33.967852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:33.967852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:33.967852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:33.967852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:33.967852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:33.967852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:33.967852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:33.967852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:33.967852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:33.967852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:33.967852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:33.967852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:33.967852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:33.968780 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:33.968780 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:33.968780 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:33.968780 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 352/717] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:33.969411 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d356eec-db18-4fb3-b1da-9fe2db58e0de None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:33.969411 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d356eec-db18-4fb3-b1da-9fe2db58e0de None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:33.970176 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d356eec-db18-4fb3-b1da-9fe2db58e0de None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:33.975636 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 353/718] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:33.975636 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d356eec-db18-4fb3-b1da-9fe2db58e0de None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:34.007300 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219964b5-5c4c-4f95-a033-98759e305a34 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:34.045753 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198f41a7-8940-4d19-b50a-4d75390fe8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:34.046380 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f41a7-8940-4d19-b50a-4d75390fe8e0 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:34.046660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f41a7-8940-4d19-b50a-4d75390fe8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:34.046964 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f41a7-8940-4d19-b50a-4d75390fe8e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:34.047645 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198f41a7-8940-4d19-b50a-4d75390fe8e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:34.047949 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198f41a7-8940-4d19-b50a-4d75390fe8e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:34.049321 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198f41a7-8940-4d19-b50a-4d75390fe8e0 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:34.058007 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 366/719] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 14:13:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:34.058736 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198f41a7-8940-4d19-b50a-4d75390fe8e0 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:34.091640 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d356eec-db18-4fb3-b1da-9fe2db58e0de None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:34.096482 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:34.130531 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba4ceb-fb66-49f2-be17-2c90bf2abc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:34.131156 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba4ceb-fb66-49f2-be17-2c90bf2abc53 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:34.131431 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba4ceb-fb66-49f2-be17-2c90bf2abc53 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:34.131695 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba4ceb-fb66-49f2-be17-2c90bf2abc53 None admin] PATH_INFO: `/v3/services` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:34.132316 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fba4ceb-fb66-49f2-be17-2c90bf2abc53 None admin] RBAC: Adding query filter params (type=network) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:34.132612 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fba4ceb-fb66-49f2-be17-2c90bf2abc53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:34.134062 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fba4ceb-fb66-49f2-be17-2c90bf2abc53 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:34.136122 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:34.140647 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 354/720] 10.222.0.52 () {56 vars in 1112 bytes} [Sat Sep 24 14:13:33 2022] GET /identity/v3/services?type=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:34.148412 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76812a7-f55e-4c03-8030-4a3836b0aa7d None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:34.149182 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76812a7-f55e-4c03-8030-4a3836b0aa7d None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:34.149459 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76812a7-f55e-4c03-8030-4a3836b0aa7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:34.167229 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198f41a7-8940-4d19-b50a-4d75390fe8e0 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:34.181359 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-b76812a7-f55e-4c03-8030-4a3836b0aa7d None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:34.182640 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-b76812a7-f55e-4c03-8030-4a3836b0aa7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:34.208211 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f021113c-f552-4c79-8cf3-ea6ba7975256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:34.209640 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021113c-f552-4c79-8cf3-ea6ba7975256 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:34.210185 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021113c-f552-4c79-8cf3-ea6ba7975256 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:34.210690 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021113c-f552-4c79-8cf3-ea6ba7975256 None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:34.211600 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f021113c-f552-4c79-8cf3-ea6ba7975256 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:34.212134 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f021113c-f552-4c79-8cf3-ea6ba7975256 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:34.214463 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f021113c-f552-4c79-8cf3-ea6ba7975256 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:34.233062 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 367/721] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:34.234441 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f021113c-f552-4c79-8cf3-ea6ba7975256 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:34.277743 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 355/722] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:13:34.284039 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76812a7-f55e-4c03-8030-4a3836b0aa7d None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:34.366146 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f021113c-f552-4c79-8cf3-ea6ba7975256 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:34.403649 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a56f71-124f-40fb-abcb-e3da24a75a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:34.404282 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a56f71-124f-40fb-abcb-e3da24a75a1d None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:34.404553 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a56f71-124f-40fb-abcb-e3da24a75a1d None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:34.404823 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a56f71-124f-40fb-abcb-e3da24a75a1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:34.407613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:34.416673 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76812a7-f55e-4c03-8030-4a3836b0aa7d None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:34.416836 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a56f71-124f-40fb-abcb-e3da24a75a1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:34.418140 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a56f71-124f-40fb-abcb-e3da24a75a1d None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:34.419100 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:34.436284 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-24a56f71-124f-40fb-abcb-e3da24a75a1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:34.436284 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:34.436284 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:34.436284 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:34.436284 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:34.436284 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:34.436284 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:34.436284 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:34.436284 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:34.436284 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:34.436284 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:34.436284 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:34.436284 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:34.436284 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:34.436284 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:34.436284 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:34.437271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:34.437271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:34.437271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:34.437271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:34.437271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:34.437271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:34.437271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:34.437271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:34.437271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:34.437271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:34.437271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:34.437271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:34.437271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:34.437271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:34.437271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:34.437271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:34.437271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:34.438775 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:34.438775 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:34.438775 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:34.438775 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 368/723] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:34.442703 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a56f71-124f-40fb-abcb-e3da24a75a1d None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:34.477716 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac3d677-9039-4525-8c4b-b244a3db9b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:34.480769 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3d677-9039-4525-8c4b-b244a3db9b87 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:34.481049 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3d677-9039-4525-8c4b-b244a3db9b87 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:34.482647 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3d677-9039-4525-8c4b-b244a3db9b87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:34.499492 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac3d677-9039-4525-8c4b-b244a3db9b87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:34.501000 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac3d677-9039-4525-8c4b-b244a3db9b87 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:34.523885 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-7ac3d677-9039-4525-8c4b-b244a3db9b87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:34.523885 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:34.523885 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:34.523885 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:34.523885 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:34.523885 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:34.523885 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:34.523885 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:34.523885 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:34.523885 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:34.523885 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:34.523885 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:34.523885 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:13:34.523885 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:13:34.523885 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:13:34.525224 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:13:34.525224 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:34.525224 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:34.525224 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:34.525224 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:34.525224 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:34.525224 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:34.525224 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:34.525224 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:34.525224 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:34.525224 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:34.525224 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:34.525224 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:34.525224 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:34.525224 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:34.525224 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:34.525224 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:34.526889 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:13:34.526889 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:13:34.526889 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:13:34.526889 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:13:34.526889 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:13:34.526889 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:13:34.526889 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:34.526889 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:34.526889 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 356/724] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:13:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:34.531269 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac3d677-9039-4525-8c4b-b244a3db9b87 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:34.617878 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a56f71-124f-40fb-abcb-e3da24a75a1d None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:34.663977 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32ace72-06ab-412c-9076-a850f7493b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:34.664933 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ace72-06ab-412c-9076-a850f7493b05 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:34.665933 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ace72-06ab-412c-9076-a850f7493b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:34.666411 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ace72-06ab-412c-9076-a850f7493b05 None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:34.667287 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32ace72-06ab-412c-9076-a850f7493b05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:34.667816 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32ace72-06ab-412c-9076-a850f7493b05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:34.669399 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32ace72-06ab-412c-9076-a850f7493b05 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:34.680634 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac3d677-9039-4525-8c4b-b244a3db9b87 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:34.686932 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 369/725] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:13:34.688011 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32ace72-06ab-412c-9076-a850f7493b05 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:34.726132 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8696db75-45bf-4f52-9920-e39ce216d67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:34.726753 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696db75-45bf-4f52-9920-e39ce216d67e None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:34.727054 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696db75-45bf-4f52-9920-e39ce216d67e None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:34.727324 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696db75-45bf-4f52-9920-e39ce216d67e None admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:34.728555 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8696db75-45bf-4f52-9920-e39ce216d67e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:34.728918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8696db75-45bf-4f52-9920-e39ce216d67e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:34.730945 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8696db75-45bf-4f52-9920-e39ce216d67e None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:34.740729 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:34.750326 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 357/726] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 14:13:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:34.751310 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8696db75-45bf-4f52-9920-e39ce216d67e None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:34.777737 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:34.802511 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32ace72-06ab-412c-9076-a850f7493b05 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:34.855545 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40544576-415a-4323-b435-c959ee9b9f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:34.856183 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40544576-415a-4323-b435-c959ee9b9f71 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:34.856463 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40544576-415a-4323-b435-c959ee9b9f71 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:34.856733 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40544576-415a-4323-b435-c959ee9b9f71 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:34.873268 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40544576-415a-4323-b435-c959ee9b9f71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:13:34.880303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-40544576-415a-4323-b435-c959ee9b9f71 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:13:34.880303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:34.880303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:34.880303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:34.880303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:34.880303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:34.880303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:34.880303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:34.880303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:34.880303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:34.880303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:34.880303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:34.880303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:13:34.880303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:13:34.880303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:13:34.880303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:13:34.881252 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:13:34.881252 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:13:34.881252 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:13:34.881252 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:13:34.881252 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:34.881252 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:34.881252 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:34.881252 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:34.881252 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:34.881252 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:34.881252 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:34.881252 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:34.881252 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:34.881252 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:34.881252 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:34.881252 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:34.882147 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:34.882147 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:34.882147 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:34.882147 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:34.882147 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:13:34.882147 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:13:34.882147 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:13:34.882147 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:13:34.882147 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:13:34.882147 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:13:34.882147 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:13:34.882147 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:34.887018 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 370/727] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 14:13:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:34.887018 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40544576-415a-4323-b435-c959ee9b9f71 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:34.901845 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8696db75-45bf-4f52-9920-e39ce216d67e None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:34.950425 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c5b63-3dd5-445c-b385-99f1b18301e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:34.951247 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c5b63-3dd5-445c-b385-99f1b18301e1 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:34.951634 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c5b63-3dd5-445c-b385-99f1b18301e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:34.952005 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c5b63-3dd5-445c-b385-99f1b18301e1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:34.962274 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566c5b63-3dd5-445c-b385-99f1b18301e1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:34.965119 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566c5b63-3dd5-445c-b385-99f1b18301e1 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:34.985984 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-566c5b63-3dd5-445c-b385-99f1b18301e1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 14:13:34.985984 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:34.985984 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:34.985984 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:34.985984 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:34.985984 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:34.985984 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:34.985984 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:34.985984 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:34.985984 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:34.985984 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:34.985984 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:34.985984 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:13:34.985984 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:13:34.985984 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:13:34.985984 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:13:34.987068 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:34.987068 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:34.987068 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:13:34.987068 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:34.987068 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:13:34.987068 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:34.987068 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:34.987068 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:34.987068 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:34.987068 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:34.987068 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:34.987068 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:34.987068 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:34.987068 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:34.987068 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:34.987068 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:34.987068 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:34.987068 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:34.988023 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:34.988023 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:34.988023 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:13:34.988023 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:13:34.988023 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:13:34.988023 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:13:34.988023 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:13:34.988023 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:13:34.988023 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 14:13:34.988023 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:34.990959 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 358/728] 10.222.0.52 () {56 vars in 1117 bytes} [Sat Sep 24 14:13:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:34.992038 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c5b63-3dd5-445c-b385-99f1b18301e1 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:35.012361 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40544576-415a-4323-b435-c959ee9b9f71 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:35.024307 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:35.052969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2ea9c8-e0c1-4cb8-b7e4-63c4201ad860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:35.053971 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ea9c8-e0c1-4cb8-b7e4-63c4201ad860 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:35.054713 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ea9c8-e0c1-4cb8-b7e4-63c4201ad860 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:35.055295 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ea9c8-e0c1-4cb8-b7e4-63c4201ad860 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:35.056812 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2ea9c8-e0c1-4cb8-b7e4-63c4201ad860 None admin] RBAC: Adding query filter params (role.id=d00b78c32dbf4cc7925a55bb621d8fcf, scope.project.id=36a3ae7bab25471fb989136150b6438c, user.id=79eaaae97aac4282a613939e5b768376) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:35.057326 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2ea9c8-e0c1-4cb8-b7e4-63c4201ad860 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:35.059117 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2ea9c8-e0c1-4cb8-b7e4-63c4201ad860 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:35.065364 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 371/729] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:13:34 2022] GET /identity/v3/role_assignments?user.id=79eaaae97aac4282a613939e5b768376&scope.project.id=36a3ae7bab25471fb989136150b6438c&role.id=d00b78c32dbf4cc7925a55bb621d8fcf => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:35.066542 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2ea9c8-e0c1-4cb8-b7e4-63c4201ad860 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:35.104687 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:35.150759 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c5b63-3dd5-445c-b385-99f1b18301e1 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:35.223841 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6fea4f-c728-4392-ae9e-cf586ef65b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:35.225063 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6fea4f-c728-4392-ae9e-cf586ef65b42 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:35.227076 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6fea4f-c728-4392-ae9e-cf586ef65b42 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:35.227687 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6fea4f-c728-4392-ae9e-cf586ef65b42 None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:35.229277 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6fea4f-c728-4392-ae9e-cf586ef65b42 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:35.230706 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6fea4f-c728-4392-ae9e-cf586ef65b42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:35.233161 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6fea4f-c728-4392-ae9e-cf586ef65b42 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:35.251056 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2ea9c8-e0c1-4cb8-b7e4-63c4201ad860 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:35.258136 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 359/730] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:13:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:35.264837 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6fea4f-c728-4392-ae9e-cf586ef65b42 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:35.324394 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ccab3a-841a-45ab-9798-a334c8be2dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:35.325009 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ccab3a-841a-45ab-9798-a334c8be2dd5 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:35.325284 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ccab3a-841a-45ab-9798-a334c8be2dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:35.325555 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ccab3a-841a-45ab-9798-a334c8be2dd5 None admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:35.326274 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ccab3a-841a-45ab-9798-a334c8be2dd5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:35.326575 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ccab3a-841a-45ab-9798-a334c8be2dd5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:35.328155 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ccab3a-841a-45ab-9798-a334c8be2dd5 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:35.349226 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 372/731] 10.222.0.52 () {56 vars in 1149 bytes} [Sat Sep 24 14:13:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:35.354670 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ccab3a-841a-45ab-9798-a334c8be2dd5 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:35.379006 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6fea4f-c728-4392-ae9e-cf586ef65b42 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:35.387330 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:35.416914 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9704dc32-f440-4312-9cf0-c36f2bb3cc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:35.417884 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704dc32-f440-4312-9cf0-c36f2bb3cc5f None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:35.418386 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704dc32-f440-4312-9cf0-c36f2bb3cc5f None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:35.418976 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704dc32-f440-4312-9cf0-c36f2bb3cc5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:35.434523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9704dc32-f440-4312-9cf0-c36f2bb3cc5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:35.436893 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9704dc32-f440-4312-9cf0-c36f2bb3cc5f None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:35.455060 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:35.456687 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-9704dc32-f440-4312-9cf0-c36f2bb3cc5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:35.456687 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:35.456687 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:35.456687 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:35.456687 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:35.456687 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:35.456687 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:35.456687 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:35.456687 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:35.456687 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:35.456687 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:35.456687 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:35.456687 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:35.456687 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:35.456687 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:35.456687 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:35.457691 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:35.457691 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:35.457691 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:35.457691 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:35.457691 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:35.457691 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:35.457691 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:35.457691 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:35.457691 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:35.457691 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:35.457691 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:35.457691 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:35.457691 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:35.457691 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:35.457691 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:35.457691 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:35.457691 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:35.458595 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:35.458595 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:35.458595 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:35.461381 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 360/732] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:13:35.464203 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5af8a-0ad5-4d15-a206-44a1c84db86a None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:35.464737 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5af8a-0ad5-4d15-a206-44a1c84db86a None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:35.465212 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5af8a-0ad5-4d15-a206-44a1c84db86a None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:35.467329 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 361/733] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:35.468529 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e5af8a-0ad5-4d15-a206-44a1c84db86a None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:35.474144 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ccab3a-841a-45ab-9798-a334c8be2dd5 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:35.510153 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c0e954-7864-4bbf-a83d-0a3bc33c740e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:35.510872 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0e954-7864-4bbf-a83d-0a3bc33c740e None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:35.510984 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0e954-7864-4bbf-a83d-0a3bc33c740e None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:35.511169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0e954-7864-4bbf-a83d-0a3bc33c740e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:35.582344 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c0e954-7864-4bbf-a83d-0a3bc33c740e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:35.583851 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c0e954-7864-4bbf-a83d-0a3bc33c740e None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:35.586321 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e5af8a-0ad5-4d15-a206-44a1c84db86a None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:35.614390 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:35.625598 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a563f502-ec91-4c52-8042-69149680b1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:35.627400 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563f502-ec91-4c52-8042-69149680b1ab None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:35.628016 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563f502-ec91-4c52-8042-69149680b1ab None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:35.628490 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563f502-ec91-4c52-8042-69149680b1ab None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:35.629367 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a563f502-ec91-4c52-8042-69149680b1ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:35.630012 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a563f502-ec91-4c52-8042-69149680b1ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:35.631823 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a563f502-ec91-4c52-8042-69149680b1ab None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:35.645965 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 362/734] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:35.647697 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38388b40-9a16-4c0c-a65c-b0fb35376d9b None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:35.648240 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38388b40-9a16-4c0c-a65c-b0fb35376d9b None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:35.648738 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38388b40-9a16-4c0c-a65c-b0fb35376d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:35.670366 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-38388b40-9a16-4c0c-a65c-b0fb35376d9b None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:35.671940 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-38388b40-9a16-4c0c-a65c-b0fb35376d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:35.697820 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 373/735] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:13:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 24 14:13:35.698971 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c0e954-7864-4bbf-a83d-0a3bc33c740e None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:35.735275 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:35.750107 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 363/736] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 14:13:35.751548 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38388b40-9a16-4c0c-a65c-b0fb35376d9b None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:35.809160 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c0e954-7864-4bbf-a83d-0a3bc33c740e None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:35.857236 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:35.865044 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38388b40-9a16-4c0c-a65c-b0fb35376d9b None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:35.872374 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4596b666-99c3-4e4d-8a49-bc82f9566057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:35.873014 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596b666-99c3-4e4d-8a49-bc82f9566057 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:35.873292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596b666-99c3-4e4d-8a49-bc82f9566057 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:35.873563 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596b666-99c3-4e4d-8a49-bc82f9566057 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:35.888013 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4596b666-99c3-4e4d-8a49-bc82f9566057 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:35.889484 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4596b666-99c3-4e4d-8a49-bc82f9566057 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:35.905533 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06daf843-636f-4a03-900e-b1f0d8870d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:35.906522 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06daf843-636f-4a03-900e-b1f0d8870d65 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:35.907119 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06daf843-636f-4a03-900e-b1f0d8870d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:35.907668 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06daf843-636f-4a03-900e-b1f0d8870d65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:35.909264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-4596b666-99c3-4e4d-8a49-bc82f9566057 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 24 14:13:35.909264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:35.909264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:35.909264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:35.909264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:35.909264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:35.909264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:35.909264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:35.909264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:35.909264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:35.909264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:35.909264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:35.909264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:13:35.909264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:13:35.909264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:13:35.909264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:13:35.910175 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:35.910175 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:35.910175 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:13:35.910175 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:35.910175 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:13:35.910175 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:35.910175 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:35.910175 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:35.910175 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:35.910175 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:35.910175 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:35.910175 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:35.910175 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:35.910175 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:35.910175 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:35.910175 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:35.910175 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:35.910175 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:35.911156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:35.911156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:35.911156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:13:35.911156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:13:35.911156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:13:35.911156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:13:35.911156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:13:35.911156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:13:35.911156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 24 14:13:35.911156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:35.915029 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 374/737] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 14:13:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:35.916444 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4596b666-99c3-4e4d-8a49-bc82f9566057 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:35.924432 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06daf843-636f-4a03-900e-b1f0d8870d65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:35.926315 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06daf843-636f-4a03-900e-b1f0d8870d65 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:35.943660 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-06daf843-636f-4a03-900e-b1f0d8870d65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:35.943660 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:35.943660 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:35.943660 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:35.943660 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:35.943660 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:35.943660 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:35.943660 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:35.943660 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:35.943660 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:35.943660 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:35.943660 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:35.943660 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:35.943660 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:35.943660 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:35.943660 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:35.944645 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:35.944645 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:35.944645 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:35.944645 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:35.944645 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:35.944645 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:35.944645 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:35.944645 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:35.944645 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:35.944645 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:35.944645 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:35.944645 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:35.944645 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:35.944645 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:35.944645 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:35.944645 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:35.944645 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:35.945520 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:35.945520 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:35.945520 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:35.947885 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 364/738] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:35.957679 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06daf843-636f-4a03-900e-b1f0d8870d65 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:36.007186 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:36.094726 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4596b666-99c3-4e4d-8a49-bc82f9566057 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:36.112182 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06daf843-636f-4a03-900e-b1f0d8870d65 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:36.150417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabac257-378d-4484-911b-2df6a8d146c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:36.151113 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabac257-378d-4484-911b-2df6a8d146c8 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:36.151396 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabac257-378d-4484-911b-2df6a8d146c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:36.151671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabac257-378d-4484-911b-2df6a8d146c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:36.152329 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabac257-378d-4484-911b-2df6a8d146c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:36.152632 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabac257-378d-4484-911b-2df6a8d146c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:36.153042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652294e1-82e4-44fd-b336-f5751557c251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:36.153660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652294e1-82e4-44fd-b336-f5751557c251 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:36.153938 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652294e1-82e4-44fd-b336-f5751557c251 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:36.154041 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabac257-378d-4484-911b-2df6a8d146c8 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:36.154211 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652294e1-82e4-44fd-b336-f5751557c251 None admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:36.154988 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652294e1-82e4-44fd-b336-f5751557c251 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:36.155297 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652294e1-82e4-44fd-b336-f5751557c251 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:36.156725 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652294e1-82e4-44fd-b336-f5751557c251 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:36.168559 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 365/739] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:36.169393 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabac257-378d-4484-911b-2df6a8d146c8 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:36.180540 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 375/740] 10.222.0.52 () {56 vars in 1135 bytes} [Sat Sep 24 14:13:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:36.181358 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652294e1-82e4-44fd-b336-f5751557c251 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:36.241517 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:36.278546 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:36.286226 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabac257-378d-4484-911b-2df6a8d146c8 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:36.301354 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652294e1-82e4-44fd-b336-f5751557c251 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:36.323758 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8463bf-061f-489a-b195-36d598900171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:36.324369 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8463bf-061f-489a-b195-36d598900171 None admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:36.324648 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8463bf-061f-489a-b195-36d598900171 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:36.324917 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8463bf-061f-489a-b195-36d598900171 None admin] PATH_INFO: `/v3/services` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:36.325559 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8463bf-061f-489a-b195-36d598900171 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:36.327145 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8463bf-061f-489a-b195-36d598900171 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:36.341671 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b2834-2cd8-482d-b775-57b26217bd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:36.342559 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b2834-2cd8-482d-b775-57b26217bd77 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:36.343055 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b2834-2cd8-482d-b775-57b26217bd77 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:36.343495 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b2834-2cd8-482d-b775-57b26217bd77 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:36.354601 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 366/741] 10.222.0.52 () {60 vars in 1141 bytes} [Sat Sep 24 14:13:36 2022] POST /identity/v3/services => generated 251 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:13:36.354601 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8463bf-061f-489a-b195-36d598900171 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:36.365102 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58b2834-2cd8-482d-b775-57b26217bd77 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:36.366607 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58b2834-2cd8-482d-b775-57b26217bd77 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:36.388261 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-d58b2834-2cd8-482d-b775-57b26217bd77 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:36.388261 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:36.388261 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:36.388261 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:36.388261 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:36.388261 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:36.388261 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:36.388261 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:36.388261 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:36.388261 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:36.388261 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:36.388261 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:36.388261 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:13:36.388261 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:13:36.388261 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:13:36.389248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:13:36.389248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:36.389248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:36.389248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:36.389248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:36.389248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:36.389248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:36.389248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:36.389248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:36.389248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:36.389248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:36.389248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:36.389248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:36.389248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:36.389248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:36.389248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:36.389248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:36.390143 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:13:36.390143 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:13:36.390143 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:13:36.390143 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:13:36.390143 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:13:36.390143 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:13:36.390143 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:36.390143 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:36.390143 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 376/742] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:13:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:36.395769 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b2834-2cd8-482d-b775-57b26217bd77 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:36.549471 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8463bf-061f-489a-b195-36d598900171 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:36.594414 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b2834-2cd8-482d-b775-57b26217bd77 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:36.595541 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e004221-aef0-4f6d-b54c-3d17ab625197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:36.596322 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e004221-aef0-4f6d-b54c-3d17ab625197 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:36.596776 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e004221-aef0-4f6d-b54c-3d17ab625197 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:36.597195 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e004221-aef0-4f6d-b54c-3d17ab625197 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:36.606005 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:36.619916 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:36.642018 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c05e2c-6391-48dc-b185-221a680c64dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:36.642645 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c05e2c-6391-48dc-b185-221a680c64dd None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:36.643775 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c05e2c-6391-48dc-b185-221a680c64dd None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:36.644067 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c05e2c-6391-48dc-b185-221a680c64dd None admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:36.644777 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c05e2c-6391-48dc-b185-221a680c64dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:36.645086 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c05e2c-6391-48dc-b185-221a680c64dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:36.647512 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c05e2c-6391-48dc-b185-221a680c64dd None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:36.660087 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 377/743] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 14:13:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:13:36.666708 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c05e2c-6391-48dc-b185-221a680c64dd None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:36.695168 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e004221-aef0-4f6d-b54c-3d17ab625197 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:36.697256 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e004221-aef0-4f6d-b54c-3d17ab625197 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:36.786413 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c05e2c-6391-48dc-b185-221a680c64dd None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:36.822124 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 367/744] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:13:36 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 14:13:36.822124 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81525db-2a74-4e67-b932-e88e947113a7 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:36.822124 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81525db-2a74-4e67-b932-e88e947113a7 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:36.822124 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81525db-2a74-4e67-b932-e88e947113a7 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:36.822501 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a2697-a58a-4ca7-ac76-17fcaa0b51df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:36.822967 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a2697-a58a-4ca7-ac76-17fcaa0b51df None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:36.823207 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 368/745] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:36.823475 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a2697-a58a-4ca7-ac76-17fcaa0b51df None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:36.823842 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a2697-a58a-4ca7-ac76-17fcaa0b51df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:36.824810 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83a2697-a58a-4ca7-ac76-17fcaa0b51df None admin] RBAC: Adding query filter params (role.id=be958e9954f04268a9f012c775fc17e5, scope.project.id=36a3ae7bab25471fb989136150b6438c, user.id=82328b7fd50649a485c6eaf27c5597c9) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:36.825219 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83a2697-a58a-4ca7-ac76-17fcaa0b51df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:36.829866 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83a2697-a58a-4ca7-ac76-17fcaa0b51df None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:36.829999 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b315dc6-48f0-43b4-80fc-3c76ec28211a None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:36.829999 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b315dc6-48f0-43b4-80fc-3c76ec28211a None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:36.829999 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b315dc6-48f0-43b4-80fc-3c76ec28211a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:36.838842 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 378/746] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:13:36 2022] GET /identity/v3/role_assignments?user.id=82328b7fd50649a485c6eaf27c5597c9&scope.project.id=36a3ae7bab25471fb989136150b6438c&role.id=be958e9954f04268a9f012c775fc17e5 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:13:36.838842 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a2697-a58a-4ca7-ac76-17fcaa0b51df None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:36.849693 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-7b315dc6-48f0-43b4-80fc-3c76ec28211a None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:36.850949 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-7b315dc6-48f0-43b4-80fc-3c76ec28211a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:36.890023 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:36.906274 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:36.935163 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 369/747] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:36 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 14:13:37.031320 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a2697-a58a-4ca7-ac76-17fcaa0b51df None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:37.082718 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c0f83-ef16-4a5f-b181-e85200f283d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:37.083532 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c0f83-ef16-4a5f-b181-e85200f283d2 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:37.083920 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c0f83-ef16-4a5f-b181-e85200f283d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:37.084294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c0f83-ef16-4a5f-b181-e85200f283d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:37.100154 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c0f83-ef16-4a5f-b181-e85200f283d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:37.101839 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c0f83-ef16-4a5f-b181-e85200f283d2 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:37.121885 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-863c0f83-ef16-4a5f-b181-e85200f283d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:37.121885 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:37.121885 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:37.121885 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:37.121885 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:37.121885 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:37.121885 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:37.121885 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:37.121885 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:37.121885 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:37.121885 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:37.121885 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:37.121885 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:37.121885 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:37.121885 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:37.121885 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:37.123872 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:37.123872 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:37.123872 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:37.123872 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:37.123872 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:37.123872 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:37.123872 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:37.123872 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:37.123872 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:37.123872 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:37.123872 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:37.123872 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:37.123872 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:37.123872 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:37.123872 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:37.123872 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:37.123872 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:37.125963 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:37.125963 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:37.125963 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:37.127541 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 379/748] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:37.135955 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b315dc6-48f0-43b4-80fc-3c76ec28211a None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:37.266661 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b315dc6-48f0-43b4-80fc-3c76ec28211a None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:37.284819 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c0f83-ef16-4a5f-b181-e85200f283d2 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:37.307164 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c748596b-1064-4414-ab71-bc7b645bc522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:37.307790 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748596b-1064-4414-ab71-bc7b645bc522 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:37.308069 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748596b-1064-4414-ab71-bc7b645bc522 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:37.308338 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748596b-1064-4414-ab71-bc7b645bc522 None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:37.308986 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c748596b-1064-4414-ab71-bc7b645bc522 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:37.309288 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c748596b-1064-4414-ab71-bc7b645bc522 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:37.311295 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c748596b-1064-4414-ab71-bc7b645bc522 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:37.325959 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 370/749] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:37.331214 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c748596b-1064-4414-ab71-bc7b645bc522 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:37.408661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:37.425265 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:37.435658 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c0f83-ef16-4a5f-b181-e85200f283d2 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:37.488663 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824cd15a-d24b-4004-b1d9-50d9ee12d815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:37.489292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824cd15a-d24b-4004-b1d9-50d9ee12d815 None admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:37.489579 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824cd15a-d24b-4004-b1d9-50d9ee12d815 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:37.489854 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824cd15a-d24b-4004-b1d9-50d9ee12d815 None admin] PATH_INFO: `/v3/services` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:37.490518 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824cd15a-d24b-4004-b1d9-50d9ee12d815 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:37.492312 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824cd15a-d24b-4004-b1d9-50d9ee12d815 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:37.514654 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 380/750] 10.222.0.52 () {60 vars in 1141 bytes} [Sat Sep 24 14:13:37 2022] POST /identity/v3/services => generated 240 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:13:37.521184 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c748596b-1064-4414-ab71-bc7b645bc522 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:37.567551 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514beb3c-b3fc-4908-ada3-2625638016ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:37.568306 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514beb3c-b3fc-4908-ada3-2625638016ac None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:37.568594 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514beb3c-b3fc-4908-ada3-2625638016ac None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:37.568868 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514beb3c-b3fc-4908-ada3-2625638016ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:37.577762 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514beb3c-b3fc-4908-ada3-2625638016ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:37.579369 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514beb3c-b3fc-4908-ada3-2625638016ac None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:37.593317 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-514beb3c-b3fc-4908-ada3-2625638016ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:37.593317 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:37.593317 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:37.593317 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:37.593317 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:37.593317 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:37.593317 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:37.593317 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:37.593317 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:37.593317 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:37.593317 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:37.593317 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:37.593317 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:13:37.593317 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:13:37.593317 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:13:37.595459 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:13:37.595459 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:37.595459 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:37.595459 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:37.595459 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:37.595459 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:37.595459 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:37.595459 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:37.595459 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:37.595459 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:37.595459 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:37.595459 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:37.595459 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:37.595459 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:37.595459 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:37.595459 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:37.595459 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:37.597556 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:13:37.597556 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:13:37.597556 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:13:37.597556 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:13:37.597556 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:13:37.597556 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:13:37.597556 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:37.597556 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:37.597556 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 371/751] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:13:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:13:37.605269 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824cd15a-d24b-4004-b1d9-50d9ee12d815 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:37.701339 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10d051-b24e-4a7b-ba19-4cfe7c2d4fc8 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:37.701518 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10d051-b24e-4a7b-ba19-4cfe7c2d4fc8 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:37.701799 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10d051-b24e-4a7b-ba19-4cfe7c2d4fc8 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:37.703205 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 372/752] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:13:37.714133 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd382663-ad70-4d7d-ae9d-392357430eb2 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:37.714444 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd382663-ad70-4d7d-ae9d-392357430eb2 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:37.714676 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd382663-ad70-4d7d-ae9d-392357430eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:37.747941 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-bd382663-ad70-4d7d-ae9d-392357430eb2 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:37.749152 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-bd382663-ad70-4d7d-ae9d-392357430eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:37.773398 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824cd15a-d24b-4004-b1d9-50d9ee12d815 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:37.816522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eb3a49-36e6-48d5-ba00-ff892ba67bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:37.817169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eb3a49-36e6-48d5-ba00-ff892ba67bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:37.817450 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eb3a49-36e6-48d5-ba00-ff892ba67bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:37.817722 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eb3a49-36e6-48d5-ba00-ff892ba67bc2 None admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:37.818419 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eb3a49-36e6-48d5-ba00-ff892ba67bc2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:37.818724 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eb3a49-36e6-48d5-ba00-ff892ba67bc2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:37.823823 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eb3a49-36e6-48d5-ba00-ff892ba67bc2 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:37.839598 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 381/753] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 14:13:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:13:37.841227 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 373/754] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:13:37.847912 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eb3a49-36e6-48d5-ba00-ff892ba67bc2 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:37.863158 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:37.954256 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eb3a49-36e6-48d5-ba00-ff892ba67bc2 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:37.989674 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3483b6d-9f2c-4b9b-b34d-8a90d2286ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:37.990287 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3483b6d-9f2c-4b9b-b34d-8a90d2286ed2 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:37.990565 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3483b6d-9f2c-4b9b-b34d-8a90d2286ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:37.990851 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3483b6d-9f2c-4b9b-b34d-8a90d2286ed2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:37.991583 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3483b6d-9f2c-4b9b-b34d-8a90d2286ed2 None admin] RBAC: Adding query filter params (role.id=d00b78c32dbf4cc7925a55bb621d8fcf, scope.project.id=36a3ae7bab25471fb989136150b6438c, user.id=879f296733114f6f844a98aaefe35e92) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:37.991889 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3483b6d-9f2c-4b9b-b34d-8a90d2286ed2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:37.993334 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3483b6d-9f2c-4b9b-b34d-8a90d2286ed2 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:38.000133 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 382/755] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:13:37 2022] GET /identity/v3/role_assignments?user.id=879f296733114f6f844a98aaefe35e92&scope.project.id=36a3ae7bab25471fb989136150b6438c&role.id=d00b78c32dbf4cc7925a55bb621d8fcf => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:38.124892 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd382663-ad70-4d7d-ae9d-392357430eb2 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:38.160273 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:38.258722 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd382663-ad70-4d7d-ae9d-392357430eb2 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:38.295849 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c8b4d-b3b5-4dd7-89e3-9f263c090a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:38.296483 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c8b4d-b3b5-4dd7-89e3-9f263c090a0b None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:38.296762 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c8b4d-b3b5-4dd7-89e3-9f263c090a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:38.297041 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c8b4d-b3b5-4dd7-89e3-9f263c090a0b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:38.308150 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c8b4d-b3b5-4dd7-89e3-9f263c090a0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:13:38.312918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-7b4c8b4d-b3b5-4dd7-89e3-9f263c090a0b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:13:38.312918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:38.312918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:38.312918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:38.312918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:38.312918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:38.312918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:38.312918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:38.312918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:38.312918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:38.312918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:38.312918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:38.312918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:13:38.312918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:13:38.312918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:13:38.312918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:13:38.313831 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:13:38.313831 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:13:38.313831 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:13:38.313831 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:13:38.313831 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:38.313831 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:38.313831 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:38.313831 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:38.313831 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:38.313831 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:38.313831 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:38.313831 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:38.313831 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:38.313831 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:38.313831 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:38.313831 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:38.314641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:38.314641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:38.314641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:38.314641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:38.314641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:13:38.314641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:13:38.314641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:13:38.314641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:13:38.314641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:13:38.314641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:13:38.314641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:13:38.314641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:38.314641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 374/756] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 14:13:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:38.321515 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3483b6d-9f2c-4b9b-b34d-8a90d2286ed2 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:38.419490 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:38.454960 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3483b6d-9f2c-4b9b-b34d-8a90d2286ed2 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:38.489929 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1163906-6b11-4dcc-bd72-ba4104de0c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:38.490605 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1163906-6b11-4dcc-bd72-ba4104de0c89 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:38.490916 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1163906-6b11-4dcc-bd72-ba4104de0c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:38.491202 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1163906-6b11-4dcc-bd72-ba4104de0c89 None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:38.491870 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1163906-6b11-4dcc-bd72-ba4104de0c89 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:38.492175 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1163906-6b11-4dcc-bd72-ba4104de0c89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:38.494093 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1163906-6b11-4dcc-bd72-ba4104de0c89 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:38.503082 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 383/757] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:13:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:38.508199 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c8b4d-b3b5-4dd7-89e3-9f263c090a0b None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:38.639923 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:38.656885 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c8b4d-b3b5-4dd7-89e3-9f263c090a0b None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:38.708650 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55174c49-135a-413b-88c2-40aafd199bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:38.709294 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55174c49-135a-413b-88c2-40aafd199bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:38.709572 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55174c49-135a-413b-88c2-40aafd199bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:38.709840 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55174c49-135a-413b-88c2-40aafd199bc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:38.723605 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55174c49-135a-413b-88c2-40aafd199bc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:38.725080 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55174c49-135a-413b-88c2-40aafd199bc5 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:38.744918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-55174c49-135a-413b-88c2-40aafd199bc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:38.744918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:38.744918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:38.744918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:38.744918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:38.744918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:38.744918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:38.744918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:38.744918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:38.744918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:38.744918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:38.744918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:38.744918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:38.744918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:38.744918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:38.744918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:38.745910 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:38.745910 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:38.745910 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:38.745910 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:38.745910 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:38.745910 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:38.745910 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:38.745910 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:38.745910 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:38.745910 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:38.745910 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:38.745910 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:38.745910 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:38.745910 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:38.745910 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:38.745910 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:38.745910 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:38.746782 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:38.746782 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:38.746782 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:38.748437 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 375/758] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:38.754956 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1163906-6b11-4dcc-bd72-ba4104de0c89 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:38.918930 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1163906-6b11-4dcc-bd72-ba4104de0c89 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:38.950341 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:38.956241 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f4258f-2ba0-4ebc-96f8-99d2bbedb3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:38.956885 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4258f-2ba0-4ebc-96f8-99d2bbedb3a8 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:38.957154 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4258f-2ba0-4ebc-96f8-99d2bbedb3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:38.957424 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4258f-2ba0-4ebc-96f8-99d2bbedb3a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:38.958079 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f4258f-2ba0-4ebc-96f8-99d2bbedb3a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:38.958389 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f4258f-2ba0-4ebc-96f8-99d2bbedb3a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:38.959846 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f4258f-2ba0-4ebc-96f8-99d2bbedb3a8 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:38.980158 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 384/759] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:38.985307 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55174c49-135a-413b-88c2-40aafd199bc5 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:39.135182 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55174c49-135a-413b-88c2-40aafd199bc5 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:39.190424 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b2c2f-cbec-459a-bf44-be56f216ceea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:39.191009 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b2c2f-cbec-459a-bf44-be56f216ceea None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:39.191286 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b2c2f-cbec-459a-bf44-be56f216ceea None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:39.191556 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b2c2f-cbec-459a-bf44-be56f216ceea None admin] PATH_INFO: `/v3/users/nova` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:39.195967 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c34306-5868-47ab-a8b6-be2713ecd895 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:39.196258 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c34306-5868-47ab-a8b6-be2713ecd895 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:39.196566 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c34306-5868-47ab-a8b6-be2713ecd895 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:39.198174 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 385/760] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:13:39.201352 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5b2c2f-cbec-459a-bf44-be56f216ceea None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:39.202796 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5b2c2f-cbec-459a-bf44-be56f216ceea None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:39.207270 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768548d-6069-44af-afd0-41c5eb30af71 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:39.207556 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768548d-6069-44af-afd0-41c5eb30af71 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:39.207830 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768548d-6069-44af-afd0-41c5eb30af71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:39.208283 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:39.218647 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-4b5b2c2f-cbec-459a-bf44-be56f216ceea None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 14:13:39.218647 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:39.218647 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:39.218647 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:39.218647 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:39.218647 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:39.218647 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:39.218647 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:39.218647 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:39.218647 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:39.218647 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:39.218647 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:39.218647 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:13:39.218647 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:13:39.218647 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:13:39.218647 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:13:39.219803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:39.219803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:39.219803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:13:39.219803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:39.219803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:13:39.219803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:39.219803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:39.219803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:39.219803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:39.219803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:39.219803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:39.219803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:39.219803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:39.219803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:39.219803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:39.219803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:39.219803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:39.219803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:39.220953 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:39.220953 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:39.220953 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:13:39.220953 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:13:39.220953 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:13:39.220953 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:13:39.220953 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:13:39.220953 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:13:39.220953 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 14:13:39.220953 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:39.227106 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 376/761] 10.222.0.52 () {56 vars in 1093 bytes} [Sat Sep 24 14:13:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:39.230762 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b2c2f-cbec-459a-bf44-be56f216ceea None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:39.247673 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-e768548d-6069-44af-afd0-41c5eb30af71 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:39.249104 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-e768548d-6069-44af-afd0-41c5eb30af71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:39.328632 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 386/762] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:13:39.393223 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b2c2f-cbec-459a-bf44-be56f216ceea None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:39.433259 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef9026-f3c8-4683-9691-9c4e4bbba85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:39.434176 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef9026-f3c8-4683-9691-9c4e4bbba85d None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:39.434626 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef9026-f3c8-4683-9691-9c4e4bbba85d None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:39.435109 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef9026-f3c8-4683-9691-9c4e4bbba85d None admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:39.436005 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ef9026-f3c8-4683-9691-9c4e4bbba85d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:39.436477 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ef9026-f3c8-4683-9691-9c4e4bbba85d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:39.438056 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ef9026-f3c8-4683-9691-9c4e4bbba85d None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:39.456132 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 377/763] 10.222.0.52 () {56 vars in 1133 bytes} [Sat Sep 24 14:13:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:39.461912 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e768548d-6069-44af-afd0-41c5eb30af71 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:39.564853 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e768548d-6069-44af-afd0-41c5eb30af71 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:39.585767 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:39.601074 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f715f7f2-248e-4253-a7e8-f234b392e427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:39.602084 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f715f7f2-248e-4253-a7e8-f234b392e427 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:39.602547 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f715f7f2-248e-4253-a7e8-f234b392e427 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:39.603562 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f715f7f2-248e-4253-a7e8-f234b392e427 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:39.613879 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f715f7f2-248e-4253-a7e8-f234b392e427 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:39.615555 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f715f7f2-248e-4253-a7e8-f234b392e427 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:39.630162 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-f715f7f2-248e-4253-a7e8-f234b392e427 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:39.630162 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:39.630162 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:39.630162 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:39.630162 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:39.630162 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:39.630162 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:39.630162 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:39.630162 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:39.630162 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:39.630162 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:39.630162 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:39.630162 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:39.630162 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:39.630162 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:39.630162 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:39.631136 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:39.631136 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:39.631136 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:39.631136 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:39.631136 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:39.631136 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:39.631136 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:39.631136 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:39.631136 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:39.631136 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:39.631136 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:39.631136 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:39.631136 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:39.631136 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:39.631136 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:39.631136 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:39.631136 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:39.632030 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:39.632030 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:39.632030 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:39.633782 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 387/764] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:39.639459 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef9026-f3c8-4683-9691-9c4e4bbba85d None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:39.698325 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:39.732955 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f715f7f2-248e-4253-a7e8-f234b392e427 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:39.747313 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef9026-f3c8-4683-9691-9c4e4bbba85d None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:39.791147 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75cf9ff-0915-482e-8401-2e6721afb858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:39.792001 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cf9ff-0915-482e-8401-2e6721afb858 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:39.792419 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cf9ff-0915-482e-8401-2e6721afb858 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:39.792824 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cf9ff-0915-482e-8401-2e6721afb858 None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:39.793629 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75cf9ff-0915-482e-8401-2e6721afb858 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:39.794105 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75cf9ff-0915-482e-8401-2e6721afb858 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:39.795640 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75cf9ff-0915-482e-8401-2e6721afb858 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:39.808180 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 378/765] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:39.813728 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75cf9ff-0915-482e-8401-2e6721afb858 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:39.920271 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75cf9ff-0915-482e-8401-2e6721afb858 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:39.929422 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f715f7f2-248e-4253-a7e8-f234b392e427 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:39.937499 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:39.959174 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e07a9d2-9c69-4d6c-b366-316236b60ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:39.960121 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07a9d2-9c69-4d6c-b366-316236b60ce8 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:39.960537 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07a9d2-9c69-4d6c-b366-316236b60ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:39.960990 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07a9d2-9c69-4d6c-b366-316236b60ce8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:39.968600 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e07a9d2-9c69-4d6c-b366-316236b60ce8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:39.970218 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e07a9d2-9c69-4d6c-b366-316236b60ce8 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:39.978722 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592828ce-48c6-462a-afc0-f9afda01d2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:39.979501 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592828ce-48c6-462a-afc0-f9afda01d2ac None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:39.979872 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592828ce-48c6-462a-afc0-f9afda01d2ac None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:39.980234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592828ce-48c6-462a-afc0-f9afda01d2ac None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:39.980908 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592828ce-48c6-462a-afc0-f9afda01d2ac None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:39.982359 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592828ce-48c6-462a-afc0-f9afda01d2ac None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:39.982852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-9e07a9d2-9c69-4d6c-b366-316236b60ce8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:39.982852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:39.982852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:39.982852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:39.982852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:39.982852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:39.982852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:39.982852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:39.982852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:39.982852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:39.982852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:39.982852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:39.982852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:13:39.982852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:13:39.982852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:13:39.983807 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:13:39.983807 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:39.983807 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:39.983807 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:39.983807 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:39.983807 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:39.983807 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:39.983807 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:39.983807 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:39.983807 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:39.983807 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:39.983807 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:39.983807 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:39.983807 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:39.983807 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:39.983807 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:39.983807 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:39.984790 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:13:39.984790 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:13:39.984790 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:13:39.984790 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:13:39.984790 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:13:39.984790 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:13:39.984790 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:39.984790 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:39.991461 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 379/766] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:13:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:39.992280 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-592828ce-48c6-462a-afc0-f9afda01d2ac None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 14:13:39.992280 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:39.992280 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:13:39.992280 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:13:39.992280 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:13:39.992280 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:13:39.992280 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:13:39.992280 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:13:39.992280 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 14:13:39.992280 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:39.992280 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:13:39.992280 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:39.992280 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:39.992280 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:39.992280 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:39.992280 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:39.992280 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:39.993237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:39.993237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:39.993237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:39.993237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:39.993237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:39.993237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:39.993237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:13:39.993237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:13:39.993237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:13:39.993237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:13:39.993237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:39.993237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:39.993237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:39.993237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:39.993237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:39.993237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:39.994100 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:39.994100 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:39.994100 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:39.994100 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:39.994100 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:39.994100 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:39.994100 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:39.994100 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:39.994100 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:39.994100 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:39.994100 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:13:39.994100 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:13:39.994100 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 14:13:39.994100 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:39.996057 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806efbb-da4c-443e-8734-2a624a81050f None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:39.996545 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806efbb-da4c-443e-8734-2a624a81050f None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:39.997005 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806efbb-da4c-443e-8734-2a624a81050f None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:39.998163 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 388/767] 10.222.0.52 () {56 vars in 1129 bytes} [Sat Sep 24 14:13:39 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:39.999029 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 380/768] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:39.999149 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592828ce-48c6-462a-afc0-f9afda01d2ac None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:40.000404 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806efbb-da4c-443e-8734-2a624a81050f None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:40.051134 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:40.127651 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592828ce-48c6-462a-afc0-f9afda01d2ac None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:40.133612 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806efbb-da4c-443e-8734-2a624a81050f None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:40.165226 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda505f-3e88-46ba-b2fc-9a664cdbbd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:40.166307 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda505f-3e88-46ba-b2fc-9a664cdbbd89 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:40.166935 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda505f-3e88-46ba-b2fc-9a664cdbbd89 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:40.167432 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda505f-3e88-46ba-b2fc-9a664cdbbd89 None admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:40.168912 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dda505f-3e88-46ba-b2fc-9a664cdbbd89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:40.169408 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dda505f-3e88-46ba-b2fc-9a664cdbbd89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:40.172112 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dda505f-3e88-46ba-b2fc-9a664cdbbd89 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:40.176114 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28838383-d303-43db-8b97-89dbad3dfe92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:40.176864 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28838383-d303-43db-8b97-89dbad3dfe92 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:40.177250 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28838383-d303-43db-8b97-89dbad3dfe92 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:40.177613 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28838383-d303-43db-8b97-89dbad3dfe92 None admin] PATH_INFO: `/v3/services` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:40.178401 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28838383-d303-43db-8b97-89dbad3dfe92 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:40.178794 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28838383-d303-43db-8b97-89dbad3dfe92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:40.180253 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28838383-d303-43db-8b97-89dbad3dfe92 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:40.189345 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 389/770] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 14:13:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:40.190958 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc5d7a-76f8-4af3-bc4e-a99273a14a6f None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:40.191368 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc5d7a-76f8-4af3-bc4e-a99273a14a6f None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:40.191814 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc5d7a-76f8-4af3-bc4e-a99273a14a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:40.192070 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 381/770] 10.222.0.52 () {56 vars in 1124 bytes} [Sat Sep 24 14:13:39 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:13:40.195387 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28838383-d303-43db-8b97-89dbad3dfe92 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:40.214071 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-55cc5d7a-76f8-4af3-bc4e-a99273a14a6f None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:40.215559 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-55cc5d7a-76f8-4af3-bc4e-a99273a14a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:40.260253 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:40.309872 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 390/771] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:13:40.311087 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc5d7a-76f8-4af3-bc4e-a99273a14a6f None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:40.344089 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:40.360719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28838383-d303-43db-8b97-89dbad3dfe92 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:40.429130 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed0926-23b4-453b-a21f-624d78a5849f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:40.430370 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed0926-23b4-453b-a21f-624d78a5849f None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:40.431908 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed0926-23b4-453b-a21f-624d78a5849f None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:40.432594 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed0926-23b4-453b-a21f-624d78a5849f None admin] PATH_INFO: `/v3/services` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:40.433485 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed0926-23b4-453b-a21f-624d78a5849f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:40.433927 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed0926-23b4-453b-a21f-624d78a5849f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:40.435652 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed0926-23b4-453b-a21f-624d78a5849f None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:40.440841 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc5d7a-76f8-4af3-bc4e-a99273a14a6f None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:40.441936 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 382/772] 10.222.0.52 () {56 vars in 1124 bytes} [Sat Sep 24 14:13:40 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:40.443484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9292bf9-f7af-4e9f-983c-8dda8e3ea381 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:40.443967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9292bf9-f7af-4e9f-983c-8dda8e3ea381 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:40.444462 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9292bf9-f7af-4e9f-983c-8dda8e3ea381 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:40.446089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 383/773] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:40.448308 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9292bf9-f7af-4e9f-983c-8dda8e3ea381 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:40.484088 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee9a9a8-397a-4c7a-9c32-121efc0d412d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:40.485557 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9a9a8-397a-4c7a-9c32-121efc0d412d None admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:40.486096 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9a9a8-397a-4c7a-9c32-121efc0d412d None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:40.486563 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9a9a8-397a-4c7a-9c32-121efc0d412d None admin] PATH_INFO: `/v3/projects/36a3ae7bab25471fb989136150b6438c/users/79eaaae97aac4282a613939e5b768376/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:40.489075 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee9a9a8-397a-4c7a-9c32-121efc0d412d None admin] RBAC: Authorizing `identity:create_grant(project_id=36a3ae7bab25471fb989136150b6438c, user_id=79eaaae97aac4282a613939e5b768376, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:40.491019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee9a9a8-397a-4c7a-9c32-121efc0d412d None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:40.507206 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 391/774] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 14:13:40 2022] PUT /identity/v3/projects/36a3ae7bab25471fb989136150b6438c/users/79eaaae97aac4282a613939e5b768376/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:13:40.509369 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2fd87a-bec6-4c1c-b90c-99f07e382d8b None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:40.510004 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2fd87a-bec6-4c1c-b90c-99f07e382d8b None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:40.510926 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2fd87a-bec6-4c1c-b90c-99f07e382d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:40.538998 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-7c2fd87a-bec6-4c1c-b90c-99f07e382d8b None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:40.540311 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-7c2fd87a-bec6-4c1c-b90c-99f07e382d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:40.581943 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9292bf9-f7af-4e9f-983c-8dda8e3ea381 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:40.603782 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:40.652274 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0a4f85-64b2-4f72-bdff-cfec249e415d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:40.653031 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a4f85-64b2-4f72-bdff-cfec249e415d None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:40.653314 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a4f85-64b2-4f72-bdff-cfec249e415d None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:40.653586 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a4f85-64b2-4f72-bdff-cfec249e415d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:40.654471 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0a4f85-64b2-4f72-bdff-cfec249e415d None admin] RBAC: Adding query filter params (interface=public, service_id=0cf25d3a390f4d91a9a0999194408f59, region_id=RegionOne) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:40.654909 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0a4f85-64b2-4f72-bdff-cfec249e415d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:40.656418 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0a4f85-64b2-4f72-bdff-cfec249e415d None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:40.660355 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 392/775] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:13:40.661774 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3495ef32-f5b3-4241-b28a-b3f51c0d7b06 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:40.662182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3495ef32-f5b3-4241-b28a-b3f51c0d7b06 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:40.662568 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3495ef32-f5b3-4241-b28a-b3f51c0d7b06 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:40.664263 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 393/776] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:40.668324 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3495ef32-f5b3-4241-b28a-b3f51c0d7b06 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:40.677513 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 384/777] 10.222.0.52 () {56 vars in 1251 bytes} [Sat Sep 24 14:13:40 2022] GET /identity/v3/endpoints?service_id=0cf25d3a390f4d91a9a0999194408f59&interface=public®ion_id=RegionOne => generated 194 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:40.678332 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4846e-15f0-4b11-8de9-ed0a3e8d7acd None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:40.678619 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4846e-15f0-4b11-8de9-ed0a3e8d7acd None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:40.678913 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4846e-15f0-4b11-8de9-ed0a3e8d7acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:40.693630 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:40.702937 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-24f4846e-15f0-4b11-8de9-ed0a3e8d7acd None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:40.704546 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-24f4846e-15f0-4b11-8de9-ed0a3e8d7acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:40.777873 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 385/778] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:13:40.779159 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f4846e-15f0-4b11-8de9-ed0a3e8d7acd None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:40.807802 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3495ef32-f5b3-4241-b28a-b3f51c0d7b06 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:40.856348 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e9ea5-74d9-43e9-a4da-a2b9b6ce6cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:40.856983 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e9ea5-74d9-43e9-a4da-a2b9b6ce6cbf None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:40.857263 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e9ea5-74d9-43e9-a4da-a2b9b6ce6cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:40.857537 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e9ea5-74d9-43e9-a4da-a2b9b6ce6cbf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:40.877144 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e9ea5-74d9-43e9-a4da-a2b9b6ce6cbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:13:40.890973 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-106e9ea5-74d9-43e9-a4da-a2b9b6ce6cbf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:13:40.890973 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:40.890973 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:40.890973 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:40.890973 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:40.890973 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:40.890973 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:40.890973 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:40.890973 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:40.890973 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:40.890973 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:40.890973 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:40.890973 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:13:40.890973 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:13:40.890973 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:13:40.890973 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:13:40.891971 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:13:40.891971 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:13:40.891971 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:13:40.891971 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:13:40.891971 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:40.891971 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:40.891971 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:40.891971 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:40.891971 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:40.891971 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:40.891971 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:40.891971 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:40.891971 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:40.891971 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:40.891971 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:40.891971 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:40.893197 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:40.893197 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:40.893197 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:40.893197 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:40.893197 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:13:40.893197 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:13:40.893197 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:13:40.893197 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:13:40.893197 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:13:40.893197 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:13:40.893197 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:13:40.893197 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:40.893197 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 394/779] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 14:13:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:13:40.900452 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e9ea5-74d9-43e9-a4da-a2b9b6ce6cbf None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:40.994314 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:40.998269 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f4846e-15f0-4b11-8de9-ed0a3e8d7acd None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:41.019272 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:41.038078 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56337876-38bd-4c33-a3a2-640f33be5450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:41.039147 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56337876-38bd-4c33-a3a2-640f33be5450 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:41.039590 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56337876-38bd-4c33-a3a2-640f33be5450 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:41.040017 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56337876-38bd-4c33-a3a2-640f33be5450 None admin] PATH_INFO: `/v3/services` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:41.040878 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56337876-38bd-4c33-a3a2-640f33be5450 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:41.042909 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56337876-38bd-4c33-a3a2-640f33be5450 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:41.045358 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e9ea5-74d9-43e9-a4da-a2b9b6ce6cbf None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:41.048971 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 386/780] 10.222.0.52 () {56 vars in 1087 bytes} [Sat Sep 24 14:13:40 2022] GET /identity/v3/services => generated 772 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:41.049960 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56337876-38bd-4c33-a3a2-640f33be5450 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:41.086426 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82845a5a-2b2b-414e-b05a-7954332d1739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:41.087093 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82845a5a-2b2b-414e-b05a-7954332d1739 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:41.087425 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82845a5a-2b2b-414e-b05a-7954332d1739 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:41.087704 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82845a5a-2b2b-414e-b05a-7954332d1739 None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:41.088373 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82845a5a-2b2b-414e-b05a-7954332d1739 None admin] RBAC: Adding query filter params (name=service) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:41.088780 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82845a5a-2b2b-414e-b05a-7954332d1739 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:41.090283 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82845a5a-2b2b-414e-b05a-7954332d1739 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:41.100227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 395/781] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 14:13:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:13:41.108005 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82845a5a-2b2b-414e-b05a-7954332d1739 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:41.168108 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56337876-38bd-4c33-a3a2-640f33be5450 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:41.212854 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e1dfc-a7d6-4ca4-a5f6-a493df7f489c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:41.213490 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e1dfc-a7d6-4ca4-a5f6-a493df7f489c None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:41.213759 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e1dfc-a7d6-4ca4-a5f6-a493df7f489c None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:41.214029 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e1dfc-a7d6-4ca4-a5f6-a493df7f489c None admin] PATH_INFO: `/v3/services/network` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:41.214732 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88e1dfc-a7d6-4ca4-a5f6-a493df7f489c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:41.216301 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88e1dfc-a7d6-4ca4-a5f6-a493df7f489c None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:41.227784 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-b88e1dfc-a7d6-4ca4-a5f6-a493df7f489c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 14:13:41.227784 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:41.227784 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:13:41.227784 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:13:41.227784 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:13:41.227784 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:13:41.227784 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:13:41.227784 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:13:41.227784 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 14:13:41.227784 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:41.227784 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:13:41.227784 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:41.227784 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:41.227784 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:41.227784 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:41.227784 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:41.227784 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:41.228840 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:41.228840 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:41.228840 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:41.228840 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:41.228840 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:41.228840 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:41.228840 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:13:41.228840 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:13:41.228840 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:13:41.228840 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:13:41.228840 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:41.228840 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:41.228840 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:41.228840 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:41.228840 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:41.228840 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:41.229656 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:41.229656 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:41.229656 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:41.229656 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:41.229656 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:41.229656 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:41.229656 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:41.229656 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:41.229656 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:41.229656 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:41.229656 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:13:41.229656 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:13:41.229656 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 14:13:41.229656 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:41.229656 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 387/782] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:13:41 2022] GET /identity/v3/services/network => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:41.230441 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e1dfc-a7d6-4ca4-a5f6-a493df7f489c None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:41.252550 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82845a5a-2b2b-414e-b05a-7954332d1739 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:41.264068 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:41.299095 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b270deda-cf6d-4ef9-aa4b-5680d628c680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:41.300058 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270deda-cf6d-4ef9-aa4b-5680d628c680 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:41.300330 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270deda-cf6d-4ef9-aa4b-5680d628c680 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:41.300860 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270deda-cf6d-4ef9-aa4b-5680d628c680 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:41.311160 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:41.316321 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b270deda-cf6d-4ef9-aa4b-5680d628c680 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:41.317893 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b270deda-cf6d-4ef9-aa4b-5680d628c680 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:41.336066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-b270deda-cf6d-4ef9-aa4b-5680d628c680 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:41.336066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:41.336066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:41.336066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:41.336066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:41.336066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:41.336066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:41.336066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:41.336066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:41.336066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:41.336066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:41.336066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:41.336066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:41.336066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:41.336066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:41.336066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:41.337784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:41.337784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:41.337784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:41.337784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:41.337784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:41.337784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:41.337784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:41.337784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:41.337784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:41.337784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:41.337784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:41.337784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:41.337784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:41.337784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:41.337784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:41.337784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:41.337784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:41.338970 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:41.338970 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:41.338970 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:41.339936 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 396/783] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:41.340731 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b270deda-cf6d-4ef9-aa4b-5680d628c680 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:41.355297 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e1dfc-a7d6-4ca4-a5f6-a493df7f489c None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:41.397054 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4fdb07-84f0-4b09-99ec-505711e99064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:41.397684 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4fdb07-84f0-4b09-99ec-505711e99064 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:41.398220 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4fdb07-84f0-4b09-99ec-505711e99064 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:41.398490 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4fdb07-84f0-4b09-99ec-505711e99064 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:41.412639 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4fdb07-84f0-4b09-99ec-505711e99064 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:13:41.419089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-7b4fdb07-84f0-4b09-99ec-505711e99064 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:13:41.419089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:41.419089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:41.419089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:41.419089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:41.419089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:41.419089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:41.419089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:41.419089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:41.419089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:41.419089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:41.419089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:41.419089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:13:41.419089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:13:41.419089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:13:41.419089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:13:41.420030 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:13:41.420030 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:13:41.420030 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:13:41.420030 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:13:41.420030 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:41.420030 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:41.420030 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:41.420030 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:41.420030 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:41.420030 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:41.420030 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:41.420030 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:41.420030 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:41.420030 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:41.420030 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:41.420030 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:41.420861 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:41.420861 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:41.420861 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:41.420861 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:41.420861 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:13:41.420861 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:13:41.420861 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:13:41.420861 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:13:41.420861 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:13:41.420861 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:13:41.420861 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:13:41.420861 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:41.423477 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 388/784] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 14:13:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:41.424464 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4fdb07-84f0-4b09-99ec-505711e99064 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:41.452969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b270deda-cf6d-4ef9-aa4b-5680d628c680 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:41.497243 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f21f0e-1b5d-43d0-894f-4cc1eba9bd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:41.497876 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f21f0e-1b5d-43d0-894f-4cc1eba9bd5a None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:41.498107 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f21f0e-1b5d-43d0-894f-4cc1eba9bd5a None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:41.498372 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f21f0e-1b5d-43d0-894f-4cc1eba9bd5a None admin] PATH_INFO: `/v3/services` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:41.499634 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f21f0e-1b5d-43d0-894f-4cc1eba9bd5a None admin] RBAC: Adding query filter params (name=network) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:41.499634 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f21f0e-1b5d-43d0-894f-4cc1eba9bd5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:41.500675 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f21f0e-1b5d-43d0-894f-4cc1eba9bd5a None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:41.510946 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 397/785] 10.222.0.52 () {56 vars in 1112 bytes} [Sat Sep 24 14:13:41 2022] GET /identity/v3/services?name=network => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:41.511687 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f21f0e-1b5d-43d0-894f-4cc1eba9bd5a None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:41.528006 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:41.544015 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4fdb07-84f0-4b09-99ec-505711e99064 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:41.579733 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:41.584858 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb6b9e9-ec92-460f-8294-3c7d2bfa1252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:41.586462 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6b9e9-ec92-460f-8294-3c7d2bfa1252 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:41.586933 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6b9e9-ec92-460f-8294-3c7d2bfa1252 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:41.587922 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6b9e9-ec92-460f-8294-3c7d2bfa1252 None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:41.588959 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb6b9e9-ec92-460f-8294-3c7d2bfa1252 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:41.589799 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb6b9e9-ec92-460f-8294-3c7d2bfa1252 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:41.591667 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb6b9e9-ec92-460f-8294-3c7d2bfa1252 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:41.606310 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 389/786] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:41.607450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb6b9e9-ec92-460f-8294-3c7d2bfa1252 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:41.634718 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f21f0e-1b5d-43d0-894f-4cc1eba9bd5a None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:41.673816 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16972b8e-3a72-411b-9a49-3866a074acf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:41.674521 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16972b8e-3a72-411b-9a49-3866a074acf8 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:41.674883 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16972b8e-3a72-411b-9a49-3866a074acf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:41.675557 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16972b8e-3a72-411b-9a49-3866a074acf8 None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:41.676285 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16972b8e-3a72-411b-9a49-3866a074acf8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:41.676650 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16972b8e-3a72-411b-9a49-3866a074acf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:41.678586 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16972b8e-3a72-411b-9a49-3866a074acf8 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:41.695085 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 398/787] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:13:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:41.696818 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16972b8e-3a72-411b-9a49-3866a074acf8 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:41.730876 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb6b9e9-ec92-460f-8294-3c7d2bfa1252 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:41.772192 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c3fb2-f5fe-41c0-9e8b-7263832019ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:41.773429 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c3fb2-f5fe-41c0-9e8b-7263832019ac None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:41.773893 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c3fb2-f5fe-41c0-9e8b-7263832019ac None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:41.774335 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c3fb2-f5fe-41c0-9e8b-7263832019ac None admin] PATH_INFO: `/v3/services` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:41.775682 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8c3fb2-f5fe-41c0-9e8b-7263832019ac None admin] RBAC: Adding query filter params (type=network) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:41.776197 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8c3fb2-f5fe-41c0-9e8b-7263832019ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:41.777831 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8c3fb2-f5fe-41c0-9e8b-7263832019ac None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:41.784186 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 390/788] 10.222.0.52 () {56 vars in 1112 bytes} [Sat Sep 24 14:13:41 2022] GET /identity/v3/services?type=network => generated 350 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:41.789950 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c3fb2-f5fe-41c0-9e8b-7263832019ac None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:41.807459 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:41.823862 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16972b8e-3a72-411b-9a49-3866a074acf8 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:41.845095 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:41.878202 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a309b9-7af3-4548-a7a8-f81a6a299ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:41.878849 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a309b9-7af3-4548-a7a8-f81a6a299ee5 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:41.879135 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a309b9-7af3-4548-a7a8-f81a6a299ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:41.879403 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a309b9-7af3-4548-a7a8-f81a6a299ee5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:41.887555 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a309b9-7af3-4548-a7a8-f81a6a299ee5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:41.888951 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a309b9-7af3-4548-a7a8-f81a6a299ee5 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:41.901904 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-22a309b9-7af3-4548-a7a8-f81a6a299ee5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 14:13:41.901904 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:41.901904 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:41.901904 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:41.901904 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:41.901904 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:41.901904 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:41.901904 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:41.901904 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:41.901904 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:41.901904 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:41.901904 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:41.901904 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:13:41.901904 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:13:41.901904 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:13:41.901904 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:13:41.902934 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:41.902934 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:41.902934 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:13:41.902934 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:41.902934 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:13:41.902934 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:41.902934 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:41.902934 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:41.902934 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:41.902934 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:41.902934 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:41.902934 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:41.902934 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:41.902934 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:41.902934 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:41.902934 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:41.902934 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:41.902934 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:41.903970 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:41.903970 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:41.903970 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:13:41.903970 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:13:41.903970 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:13:41.903970 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:13:41.903970 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:13:41.903970 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:13:41.903970 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 14:13:41.903970 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:41.909068 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 399/789] 10.222.0.52 () {56 vars in 1117 bytes} [Sat Sep 24 14:13:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:41.910413 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a309b9-7af3-4548-a7a8-f81a6a299ee5 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:41.935714 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c3fb2-f5fe-41c0-9e8b-7263832019ac None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:41.981908 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71ce6b-1026-4757-8aab-e96f9d8653b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:41.982922 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71ce6b-1026-4757-8aab-e96f9d8653b0 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:41.983424 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71ce6b-1026-4757-8aab-e96f9d8653b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:41.984446 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71ce6b-1026-4757-8aab-e96f9d8653b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:41.997017 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e71ce6b-1026-4757-8aab-e96f9d8653b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:41.998589 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e71ce6b-1026-4757-8aab-e96f9d8653b0 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:42.013958 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-3e71ce6b-1026-4757-8aab-e96f9d8653b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:42.013958 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:42.013958 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:42.013958 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:42.013958 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:42.013958 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:42.013958 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:42.013958 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:42.013958 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:42.013958 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:42.013958 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:42.013958 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:42.013958 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:42.013958 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:42.013958 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:42.013958 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:42.014960 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:42.014960 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:42.014960 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:42.014960 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:42.014960 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:42.014960 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:42.014960 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:42.014960 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:42.014960 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:42.014960 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:42.014960 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:42.014960 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:42.014960 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:42.014960 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:42.014960 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:42.014960 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:42.014960 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:42.015859 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:42.015859 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:42.015859 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:42.019737 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 391/790] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:42.020570 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71ce6b-1026-4757-8aab-e96f9d8653b0 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:42.051036 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a309b9-7af3-4548-a7a8-f81a6a299ee5 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:42.084882 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:42.102300 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6570d3-3a35-448e-a8ef-b7e6cfbdd93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:42.104387 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6570d3-3a35-448e-a8ef-b7e6cfbdd93e None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:42.104875 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6570d3-3a35-448e-a8ef-b7e6cfbdd93e None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:42.105852 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6570d3-3a35-448e-a8ef-b7e6cfbdd93e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:42.107106 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6570d3-3a35-448e-a8ef-b7e6cfbdd93e None admin] RBAC: Adding query filter params (interface=public, service_id=1b57386174aa476fb5dc109e01f0bd54, region_id=RegionOne) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:42.107236 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6570d3-3a35-448e-a8ef-b7e6cfbdd93e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:42.108612 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6570d3-3a35-448e-a8ef-b7e6cfbdd93e None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:42.117118 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 400/791] 10.222.0.52 () {56 vars in 1251 bytes} [Sat Sep 24 14:13:41 2022] GET /identity/v3/endpoints?service_id=1b57386174aa476fb5dc109e01f0bd54&interface=public®ion_id=RegionOne => generated 194 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:42.117949 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6570d3-3a35-448e-a8ef-b7e6cfbdd93e None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:42.140975 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71ce6b-1026-4757-8aab-e96f9d8653b0 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:42.163111 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:42.190233 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc6a401-9d5c-4531-ae32-723a6802e435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:42.191718 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc6a401-9d5c-4531-ae32-723a6802e435 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:42.192468 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc6a401-9d5c-4531-ae32-723a6802e435 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:42.192940 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc6a401-9d5c-4531-ae32-723a6802e435 None admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:42.194961 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc6a401-9d5c-4531-ae32-723a6802e435 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:42.195300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc6a401-9d5c-4531-ae32-723a6802e435 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:42.197294 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc6a401-9d5c-4531-ae32-723a6802e435 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:42.230914 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6570d3-3a35-448e-a8ef-b7e6cfbdd93e None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:42.269865 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb4a4b9-10cc-4ad8-8883-92775ed55af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:42.270905 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4a4b9-10cc-4ad8-8883-92775ed55af2 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:42.271386 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4a4b9-10cc-4ad8-8883-92775ed55af2 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:42.271852 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4a4b9-10cc-4ad8-8883-92775ed55af2 None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:42.272723 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb4a4b9-10cc-4ad8-8883-92775ed55af2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:42.273223 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb4a4b9-10cc-4ad8-8883-92775ed55af2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:42.274904 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb4a4b9-10cc-4ad8-8883-92775ed55af2 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:42.292273 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 401/792] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:42.293552 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb4a4b9-10cc-4ad8-8883-92775ed55af2 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:42.363323 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 392/793] 10.222.0.52 () {56 vars in 1149 bytes} [Sat Sep 24 14:13:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:42.363960 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc6a401-9d5c-4531-ae32-723a6802e435 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:42.414860 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:42.425687 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb4a4b9-10cc-4ad8-8883-92775ed55af2 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:42.471881 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7187743-d50f-4e10-8e90-5c48762c81b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:42.472741 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7187743-d50f-4e10-8e90-5c48762c81b5 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:42.473690 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7187743-d50f-4e10-8e90-5c48762c81b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:42.474116 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7187743-d50f-4e10-8e90-5c48762c81b5 None admin] PATH_INFO: `/v3/services` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:42.475201 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7187743-d50f-4e10-8e90-5c48762c81b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:42.477527 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7187743-d50f-4e10-8e90-5c48762c81b5 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:42.485783 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 402/794] 10.222.0.52 () {56 vars in 1087 bytes} [Sat Sep 24 14:13:42 2022] GET /identity/v3/services => generated 772 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:42.486847 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7187743-d50f-4e10-8e90-5c48762c81b5 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:42.488033 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc6a401-9d5c-4531-ae32-723a6802e435 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:42.513027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:42.531709 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66837066-d795-4607-a898-06232bca283b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:42.532931 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66837066-d795-4607-a898-06232bca283b None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:42.533565 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66837066-d795-4607-a898-06232bca283b None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:42.534089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66837066-d795-4607-a898-06232bca283b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:42.543682 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66837066-d795-4607-a898-06232bca283b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:42.545329 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66837066-d795-4607-a898-06232bca283b None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:42.558270 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-66837066-d795-4607-a898-06232bca283b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 24 14:13:42.558270 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:42.558270 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:42.558270 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:42.558270 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:42.558270 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:42.558270 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:42.558270 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:42.558270 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:42.558270 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:42.558270 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:42.558270 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:42.558270 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:13:42.558270 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:13:42.558270 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:13:42.558270 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:13:42.559520 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:42.559520 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:42.559520 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:13:42.559520 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:42.559520 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:13:42.559520 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:42.559520 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:42.559520 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:42.559520 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:42.559520 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:42.559520 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:42.559520 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:42.559520 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:42.559520 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:42.559520 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:42.559520 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:42.559520 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:42.559520 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:42.560686 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:42.560686 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:42.560686 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:13:42.560686 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:13:42.560686 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:13:42.560686 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:13:42.560686 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:13:42.560686 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:13:42.560686 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 24 14:13:42.560686 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:42.562458 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 393/795] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 14:13:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:42.567824 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66837066-d795-4607-a898-06232bca283b None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:42.653115 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7187743-d50f-4e10-8e90-5c48762c81b5 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:42.731492 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4aa35d-58e6-418f-bc8a-cb2ac45a2ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:42.731745 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4aa35d-58e6-418f-bc8a-cb2ac45a2ccf None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:42.731981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4aa35d-58e6-418f-bc8a-cb2ac45a2ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:42.732157 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4aa35d-58e6-418f-bc8a-cb2ac45a2ccf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:42.747365 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4aa35d-58e6-418f-bc8a-cb2ac45a2ccf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:42.749174 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4aa35d-58e6-418f-bc8a-cb2ac45a2ccf None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:42.762430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66837066-d795-4607-a898-06232bca283b None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:42.773315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-be4aa35d-58e6-418f-bc8a-cb2ac45a2ccf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:42.773315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:42.773315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:42.773315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:42.773315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:42.773315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:42.773315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:42.773315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:42.773315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:42.773315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:42.773315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:42.773315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:42.773315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:42.773315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:42.773315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:42.773315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:42.774275 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:42.774275 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:42.774275 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:42.774275 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:42.774275 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:42.774275 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:42.774275 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:42.774275 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:42.774275 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:42.774275 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:42.774275 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:42.774275 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:42.774275 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:42.774275 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:42.774275 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:42.774275 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:42.774275 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:42.776171 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:42.776171 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:42.776171 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:42.776171 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 403/796] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:42.780844 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4aa35d-58e6-418f-bc8a-cb2ac45a2ccf None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:42.806845 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:42.808051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da3d01d-04c6-4bec-aaf6-3b13a1c75768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:42.808946 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3d01d-04c6-4bec-aaf6-3b13a1c75768 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:42.809424 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3d01d-04c6-4bec-aaf6-3b13a1c75768 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:42.809892 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3d01d-04c6-4bec-aaf6-3b13a1c75768 None admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:42.811637 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da3d01d-04c6-4bec-aaf6-3b13a1c75768 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:42.812181 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da3d01d-04c6-4bec-aaf6-3b13a1c75768 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:42.813950 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da3d01d-04c6-4bec-aaf6-3b13a1c75768 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:42.840588 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 394/797] 10.222.0.52 () {56 vars in 1135 bytes} [Sat Sep 24 14:13:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:42.846565 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da3d01d-04c6-4bec-aaf6-3b13a1c75768 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:42.899513 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:42.911532 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4aa35d-58e6-418f-bc8a-cb2ac45a2ccf None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:42.956460 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be3df95-996e-43df-9f18-eefdd2b20d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:42.957702 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3df95-996e-43df-9f18-eefdd2b20d3a None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:42.958192 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3df95-996e-43df-9f18-eefdd2b20d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:42.958798 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3df95-996e-43df-9f18-eefdd2b20d3a None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:42.960169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be3df95-996e-43df-9f18-eefdd2b20d3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:42.960684 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be3df95-996e-43df-9f18-eefdd2b20d3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:42.963320 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be3df95-996e-43df-9f18-eefdd2b20d3a None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:42.972768 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da3d01d-04c6-4bec-aaf6-3b13a1c75768 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:42.979175 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 404/798] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:42.980774 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76751b-66ed-45c1-b349-2ef729dae931 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:42.981268 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76751b-66ed-45c1-b349-2ef729dae931 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:42.982180 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76751b-66ed-45c1-b349-2ef729dae931 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:42.984193 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 405/799] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:42.985114 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76751b-66ed-45c1-b349-2ef729dae931 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:43.018500 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50527ed6-6fce-4c37-a3bc-f8e151411d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:43.019369 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50527ed6-6fce-4c37-a3bc-f8e151411d45 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:43.019648 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50527ed6-6fce-4c37-a3bc-f8e151411d45 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:43.019920 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50527ed6-6fce-4c37-a3bc-f8e151411d45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:43.032733 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50527ed6-6fce-4c37-a3bc-f8e151411d45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:43.034656 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50527ed6-6fce-4c37-a3bc-f8e151411d45 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:43.050973 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-50527ed6-6fce-4c37-a3bc-f8e151411d45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:43.050973 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:43.050973 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:43.050973 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:43.050973 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:43.050973 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:43.050973 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:43.050973 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:43.050973 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:43.050973 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:43.050973 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:43.050973 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:43.050973 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:43.050973 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:43.050973 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:43.050973 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:43.051967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:43.051967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:43.051967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:43.051967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:43.051967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:43.051967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:43.051967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:43.051967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:43.051967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:43.051967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:43.051967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:43.051967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:43.051967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:43.051967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:43.051967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:43.051967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:43.051967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:43.052870 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:43.052870 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:43.052870 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:43.052870 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 395/800] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:43.053608 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc96c3ee-9865-44f5-898f-29e67976589e None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:43.053887 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc96c3ee-9865-44f5-898f-29e67976589e None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:43.054150 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc96c3ee-9865-44f5-898f-29e67976589e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:43.077624 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-bc96c3ee-9865-44f5-898f-29e67976589e None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:43.078802 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-bc96c3ee-9865-44f5-898f-29e67976589e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:43.105225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:43.135911 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76751b-66ed-45c1-b349-2ef729dae931 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:43.171535 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c02fcf-612c-405c-ba4d-ba04772fb913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:43.172079 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c02fcf-612c-405c-ba4d-ba04772fb913 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:43.172371 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c02fcf-612c-405c-ba4d-ba04772fb913 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:43.172624 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c02fcf-612c-405c-ba4d-ba04772fb913 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:43.173958 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 396/801] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:13:43.175224 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc96c3ee-9865-44f5-898f-29e67976589e None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:43.191077 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c02fcf-612c-405c-ba4d-ba04772fb913 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:43.192551 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c02fcf-612c-405c-ba4d-ba04772fb913 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:43.199762 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:43.217494 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-28c02fcf-612c-405c-ba4d-ba04772fb913 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:43.217494 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:43.217494 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:43.217494 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:43.217494 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:43.217494 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:43.217494 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:43.217494 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:43.217494 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:43.217494 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:43.217494 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:43.217494 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:43.217494 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:13:43.217494 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:13:43.217494 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:13:43.218653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:13:43.218653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:43.218653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:43.218653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:43.218653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:43.218653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:43.218653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:43.218653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:43.218653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:43.218653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:43.218653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:43.218653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:43.218653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:43.218653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:43.218653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:43.218653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:43.218653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:43.219699 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:13:43.219699 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:13:43.219699 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:13:43.219699 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:13:43.219699 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:13:43.219699 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:13:43.219699 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:43.219699 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:43.231680 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 406/802] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:13:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:43.238704 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c02fcf-612c-405c-ba4d-ba04772fb913 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:43.295340 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc96c3ee-9865-44f5-898f-29e67976589e None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:43.336874 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9dbc3d-be8f-478a-9a1d-d2dac3a6d4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:43.339715 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9dbc3d-be8f-478a-9a1d-d2dac3a6d4fe None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:43.340242 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9dbc3d-be8f-478a-9a1d-d2dac3a6d4fe None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:43.340835 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9dbc3d-be8f-478a-9a1d-d2dac3a6d4fe None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:43.341806 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9dbc3d-be8f-478a-9a1d-d2dac3a6d4fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:43.342313 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9dbc3d-be8f-478a-9a1d-d2dac3a6d4fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:43.344248 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9dbc3d-be8f-478a-9a1d-d2dac3a6d4fe None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:43.359781 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 397/803] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:43.367059 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9dbc3d-be8f-478a-9a1d-d2dac3a6d4fe None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:43.435270 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c02fcf-612c-405c-ba4d-ba04772fb913 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:43.469300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:43.492065 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9dbc3d-be8f-478a-9a1d-d2dac3a6d4fe None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:43.508481 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d77766-0cbd-4f19-bf96-af2a81126669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:43.509662 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d77766-0cbd-4f19-bf96-af2a81126669 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:43.509947 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d77766-0cbd-4f19-bf96-af2a81126669 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:43.510276 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d77766-0cbd-4f19-bf96-af2a81126669 None admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:43.510940 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d77766-0cbd-4f19-bf96-af2a81126669 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:43.511249 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d77766-0cbd-4f19-bf96-af2a81126669 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:43.512715 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d77766-0cbd-4f19-bf96-af2a81126669 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:43.515573 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:43.535235 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f5a90-078c-4f97-9775-aa6de889e08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:43.536208 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f5a90-078c-4f97-9775-aa6de889e08d None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:43.536634 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 407/804] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 14:13:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:43.536995 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f5a90-078c-4f97-9775-aa6de889e08d None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:43.537478 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d77766-0cbd-4f19-bf96-af2a81126669 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:43.537597 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f5a90-078c-4f97-9775-aa6de889e08d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:43.551647 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8f5a90-078c-4f97-9775-aa6de889e08d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:43.553570 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8f5a90-078c-4f97-9775-aa6de889e08d None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:43.565735 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-5b8f5a90-078c-4f97-9775-aa6de889e08d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:43.565735 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:43.565735 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:43.565735 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:43.565735 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:43.565735 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:43.565735 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:43.565735 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:43.565735 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:43.565735 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:43.565735 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:43.565735 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:43.565735 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:13:43.565735 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:13:43.565735 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:13:43.566611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:13:43.566611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:43.566611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:43.566611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:43.566611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:43.566611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:43.566611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:43.566611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:43.566611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:43.566611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:43.566611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:43.566611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:43.566611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:43.566611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:43.566611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:43.566611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:43.566611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:43.567554 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:13:43.567554 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:13:43.567554 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:13:43.567554 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:13:43.567554 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:13:43.567554 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:13:43.567554 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:43.567554 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:43.567554 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 398/805] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:13:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:43.567554 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f5a90-078c-4f97-9775-aa6de889e08d None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:43.654895 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d77766-0cbd-4f19-bf96-af2a81126669 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:43.677259 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f5a90-078c-4f97-9775-aa6de889e08d None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:43.690495 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ccc92-ae57-4650-accd-45b2e0ad4081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:43.691087 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ccc92-ae57-4650-accd-45b2e0ad4081 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:43.691365 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ccc92-ae57-4650-accd-45b2e0ad4081 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:43.691638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ccc92-ae57-4650-accd-45b2e0ad4081 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:43.701908 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44ccc92-ae57-4650-accd-45b2e0ad4081 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:13:43.706667 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-a44ccc92-ae57-4650-accd-45b2e0ad4081 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:13:43.706667 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:43.706667 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:43.706667 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:43.706667 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:43.706667 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:43.706667 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:43.706667 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:43.706667 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:43.706667 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:43.706667 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:43.706667 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:43.706667 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:13:43.706667 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:13:43.706667 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:13:43.706667 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:13:43.707746 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:13:43.707746 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:13:43.707746 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:13:43.707746 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:13:43.707746 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:43.707746 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:43.707746 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:43.707746 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:43.707746 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:43.707746 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:43.707746 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:43.707746 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:43.707746 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:43.707746 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:43.707746 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:43.707746 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:43.708585 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:43.708585 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:43.708585 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:43.708585 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:43.708585 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:13:43.708585 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:13:43.708585 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:13:43.708585 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:13:43.708585 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:13:43.708585 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:13:43.708585 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:13:43.708585 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:43.708585 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 408/806] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 14:13:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:43.709265 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ab6a56-7f27-44dc-b303-4bb37f2fc769 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:43.709265 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ab6a56-7f27-44dc-b303-4bb37f2fc769 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:43.709958 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ab6a56-7f27-44dc-b303-4bb37f2fc769 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:43.714855 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 409/807] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:43.715627 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ab6a56-7f27-44dc-b303-4bb37f2fc769 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:43.718225 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb1df0-3720-4092-a108-6c7596a87bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:43.718916 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb1df0-3720-4092-a108-6c7596a87bdf None admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:43.719196 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb1df0-3720-4092-a108-6c7596a87bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:43.719469 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb1df0-3720-4092-a108-6c7596a87bdf None admin] PATH_INFO: `/v3/projects/36a3ae7bab25471fb989136150b6438c/users/82328b7fd50649a485c6eaf27c5597c9/roles/be958e9954f04268a9f012c775fc17e5` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:43.731319 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:43.736003 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb1df0-3720-4092-a108-6c7596a87bdf None admin] RBAC: Authorizing `identity:create_grant(project_id=36a3ae7bab25471fb989136150b6438c, user_id=82328b7fd50649a485c6eaf27c5597c9, role_id=be958e9954f04268a9f012c775fc17e5)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:43.737858 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb1df0-3720-4092-a108-6c7596a87bdf None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:43.758843 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 399/808] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 14:13:43 2022] PUT /identity/v3/projects/36a3ae7bab25471fb989136150b6438c/users/82328b7fd50649a485c6eaf27c5597c9/roles/be958e9954f04268a9f012c775fc17e5 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:13:43.758843 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb1df0-3720-4092-a108-6c7596a87bdf None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:43.789219 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:43.849844 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ab6a56-7f27-44dc-b303-4bb37f2fc769 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:43.890846 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec8cfe-3245-4786-968f-f35801998c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:43.891617 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec8cfe-3245-4786-968f-f35801998c6e None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:43.891898 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec8cfe-3245-4786-968f-f35801998c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:43.892174 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec8cfe-3245-4786-968f-f35801998c6e None admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:43.892872 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec8cfe-3245-4786-968f-f35801998c6e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:43.893178 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec8cfe-3245-4786-968f-f35801998c6e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:43.894649 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec8cfe-3245-4786-968f-f35801998c6e None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:43.901312 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb1df0-3720-4092-a108-6c7596a87bdf None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:43.912925 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 410/809] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 14:13:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:43.912925 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09791620-3e86-4007-b6d3-be514c1240fa None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:43.912925 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09791620-3e86-4007-b6d3-be514c1240fa None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:43.912925 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09791620-3e86-4007-b6d3-be514c1240fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:43.932453 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-09791620-3e86-4007-b6d3-be514c1240fa None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:43.933611 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-09791620-3e86-4007-b6d3-be514c1240fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:43.960742 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499f7d1-29c7-4481-9a52-3ee58b32322c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:43.962786 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499f7d1-29c7-4481-9a52-3ee58b32322c None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:43.963102 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499f7d1-29c7-4481-9a52-3ee58b32322c None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:43.963370 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499f7d1-29c7-4481-9a52-3ee58b32322c None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:43.964397 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499f7d1-29c7-4481-9a52-3ee58b32322c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:43.964676 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499f7d1-29c7-4481-9a52-3ee58b32322c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:43.967099 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499f7d1-29c7-4481-9a52-3ee58b32322c None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:43.982157 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 400/810] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:13:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:43.983078 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499f7d1-29c7-4481-9a52-3ee58b32322c None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:44.014216 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 411/811] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:13:44.019457 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09791620-3e86-4007-b6d3-be514c1240fa None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:44.060484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:44.104514 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:44.135981 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499f7d1-29c7-4481-9a52-3ee58b32322c None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:44.207024 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2cb9e-e052-4817-8726-c441b89ffc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:44.207686 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2cb9e-e052-4817-8726-c441b89ffc04 None admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:44.207958 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2cb9e-e052-4817-8726-c441b89ffc04 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:44.208226 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2cb9e-e052-4817-8726-c441b89ffc04 None admin] PATH_INFO: `/v3/projects/36a3ae7bab25471fb989136150b6438c/users/879f296733114f6f844a98aaefe35e92/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:44.210735 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09791620-3e86-4007-b6d3-be514c1240fa None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:44.214872 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a2cb9e-e052-4817-8726-c441b89ffc04 None admin] RBAC: Authorizing `identity:create_grant(project_id=36a3ae7bab25471fb989136150b6438c, user_id=879f296733114f6f844a98aaefe35e92, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:44.216249 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a2cb9e-e052-4817-8726-c441b89ffc04 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:44.244171 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 401/812] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 14:13:43 2022] PUT /identity/v3/projects/36a3ae7bab25471fb989136150b6438c/users/879f296733114f6f844a98aaefe35e92/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:13:44.264183 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5bcb6-1bf2-44cc-9c8f-db32fcc2d0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:44.264827 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5bcb6-1bf2-44cc-9c8f-db32fcc2d0ef None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:44.265102 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5bcb6-1bf2-44cc-9c8f-db32fcc2d0ef None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:44.265368 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5bcb6-1bf2-44cc-9c8f-db32fcc2d0ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:44.276686 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd5bcb6-1bf2-44cc-9c8f-db32fcc2d0ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:44.284673 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd5bcb6-1bf2-44cc-9c8f-db32fcc2d0ef None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:44.300666 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-cdd5bcb6-1bf2-44cc-9c8f-db32fcc2d0ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:44.300666 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:44.300666 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:44.300666 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:44.300666 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:44.300666 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:44.300666 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:44.300666 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:44.300666 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:44.300666 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:44.300666 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:44.300666 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:44.300666 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:44.300666 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:44.300666 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:44.300666 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:44.301659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:44.301659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:44.301659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:44.301659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:44.301659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:44.301659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:44.301659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:44.301659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:44.301659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:44.301659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:44.301659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:44.301659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:44.301659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:44.301659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:44.301659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:44.301659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:44.301659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:44.302549 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:44.302549 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:44.302549 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:44.307398 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 412/813] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:44.313127 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2cb9e-e052-4817-8726-c441b89ffc04 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:44.346713 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5bcb6-1bf2-44cc-9c8f-db32fcc2d0ef None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:44.444655 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2cb9e-e052-4817-8726-c441b89ffc04 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:44.461674 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:44.489071 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70bb08e-490d-4d68-851b-42ab8d105c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:44.490192 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70bb08e-490d-4d68-851b-42ab8d105c21 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:44.490555 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70bb08e-490d-4d68-851b-42ab8d105c21 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:44.490933 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70bb08e-490d-4d68-851b-42ab8d105c21 None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:44.491672 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70bb08e-490d-4d68-851b-42ab8d105c21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:44.492050 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70bb08e-490d-4d68-851b-42ab8d105c21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:44.494175 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70bb08e-490d-4d68-851b-42ab8d105c21 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:44.508456 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 402/814] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:44.517191 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70bb08e-490d-4d68-851b-42ab8d105c21 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:44.560541 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:44.565113 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5bcb6-1bf2-44cc-9c8f-db32fcc2d0ef None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:44.603274 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078735f9-bbc9-4e56-8aab-67c965b04908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:44.604307 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078735f9-bbc9-4e56-8aab-67c965b04908 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:44.604784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078735f9-bbc9-4e56-8aab-67c965b04908 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:44.605237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078735f9-bbc9-4e56-8aab-67c965b04908 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:44.605989 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078735f9-bbc9-4e56-8aab-67c965b04908 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:44.607726 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078735f9-bbc9-4e56-8aab-67c965b04908 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:44.617378 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-078735f9-bbc9-4e56-8aab-67c965b04908 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 14:13:44.617378 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:44.617378 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:13:44.617378 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:13:44.617378 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:13:44.617378 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:13:44.617378 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:13:44.617378 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:13:44.617378 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 14:13:44.617378 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:44.617378 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:13:44.617378 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:44.617378 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:44.617378 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:44.617378 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:44.617378 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:44.617378 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:44.618425 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:44.618425 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:44.618425 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:44.618425 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:44.618425 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:44.618425 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:44.618425 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:13:44.618425 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:13:44.618425 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:13:44.618425 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:13:44.618425 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:44.618425 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:44.618425 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:44.618425 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:44.618425 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:44.618425 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:44.619292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:44.619292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:44.619292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:44.619292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:44.619292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:44.619292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:44.619292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:44.619292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:44.619292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:44.619292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:44.619292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:13:44.619292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:13:44.619292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 14:13:44.619292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:44.621847 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 413/815] 10.222.0.52 () {56 vars in 1129 bytes} [Sat Sep 24 14:13:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:44.628230 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078735f9-bbc9-4e56-8aab-67c965b04908 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:44.636395 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70bb08e-490d-4d68-851b-42ab8d105c21 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:44.677029 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa64b55c-8a46-4d04-8f83-c1bab21b8d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:44.677447 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa64b55c-8a46-4d04-8f83-c1bab21b8d47 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:44.677729 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa64b55c-8a46-4d04-8f83-c1bab21b8d47 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:44.677997 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa64b55c-8a46-4d04-8f83-c1bab21b8d47 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:44.686100 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa64b55c-8a46-4d04-8f83-c1bab21b8d47 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:44.687741 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa64b55c-8a46-4d04-8f83-c1bab21b8d47 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:44.701447 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-aa64b55c-8a46-4d04-8f83-c1bab21b8d47 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 14:13:44.701447 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:44.701447 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:44.701447 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:44.701447 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:44.701447 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:44.701447 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:44.701447 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:44.701447 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:44.701447 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:44.701447 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:44.701447 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:44.701447 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:13:44.701447 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:13:44.701447 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:13:44.701447 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:13:44.702523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:44.702523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:44.702523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:13:44.702523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:44.702523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:13:44.702523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:44.702523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:44.702523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:44.702523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:44.702523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:44.702523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:44.702523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:44.702523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:44.702523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:44.702523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:44.702523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:44.702523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:44.702523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:44.703574 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:44.703574 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:44.703574 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:13:44.703574 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:13:44.703574 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:13:44.703574 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:13:44.703574 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:13:44.703574 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:13:44.703574 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 14:13:44.703574 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:44.703574 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 403/816] 10.222.0.52 () {56 vars in 1093 bytes} [Sat Sep 24 14:13:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:13:44.708223 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa64b55c-8a46-4d04-8f83-c1bab21b8d47 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:44.744711 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:44.768502 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078735f9-bbc9-4e56-8aab-67c965b04908 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:44.807026 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f417ebcc-5739-4d50-889d-42426712f815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:44.807704 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417ebcc-5739-4d50-889d-42426712f815 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:44.808009 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417ebcc-5739-4d50-889d-42426712f815 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:44.808515 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417ebcc-5739-4d50-889d-42426712f815 None admin] PATH_INFO: `/v3/services` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:44.809044 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f417ebcc-5739-4d50-889d-42426712f815 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:44.809400 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f417ebcc-5739-4d50-889d-42426712f815 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:44.810895 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f417ebcc-5739-4d50-889d-42426712f815 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:44.817930 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 414/817] 10.222.0.52 () {56 vars in 1124 bytes} [Sat Sep 24 14:13:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:13:44.826568 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f417ebcc-5739-4d50-889d-42426712f815 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:44.845556 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa64b55c-8a46-4d04-8f83-c1bab21b8d47 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:44.861606 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:44.883864 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c153001-002a-4bfe-ba0c-8b2e31c4a77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:44.885046 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c153001-002a-4bfe-ba0c-8b2e31c4a77f None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:44.885327 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c153001-002a-4bfe-ba0c-8b2e31c4a77f None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:44.885607 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c153001-002a-4bfe-ba0c-8b2e31c4a77f None admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:44.886342 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c153001-002a-4bfe-ba0c-8b2e31c4a77f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:44.886738 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c153001-002a-4bfe-ba0c-8b2e31c4a77f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:44.888152 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c153001-002a-4bfe-ba0c-8b2e31c4a77f None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:44.906980 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 404/818] 10.222.0.52 () {56 vars in 1133 bytes} [Sat Sep 24 14:13:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:44.913011 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c153001-002a-4bfe-ba0c-8b2e31c4a77f None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:44.941944 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f417ebcc-5739-4d50-889d-42426712f815 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:44.980418 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc691223-5617-4b0b-8198-636493b6b5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:44.981308 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc691223-5617-4b0b-8198-636493b6b5c3 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:44.981727 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc691223-5617-4b0b-8198-636493b6b5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:44.982138 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc691223-5617-4b0b-8198-636493b6b5c3 None admin] PATH_INFO: `/v3/services` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:44.982931 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc691223-5617-4b0b-8198-636493b6b5c3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:44.983373 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc691223-5617-4b0b-8198-636493b6b5c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:44.984899 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc691223-5617-4b0b-8198-636493b6b5c3 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:44.992613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 415/819] 10.222.0.52 () {56 vars in 1124 bytes} [Sat Sep 24 14:13:44 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:44.994607 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4bb865-5ec8-4af6-9c30-c0130a00e6ca None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:44.995124 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4bb865-5ec8-4af6-9c30-c0130a00e6ca None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:44.995538 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4bb865-5ec8-4af6-9c30-c0130a00e6ca None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:44.997135 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 416/820] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:44.998998 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4bb865-5ec8-4af6-9c30-c0130a00e6ca None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:45.021161 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:45.031921 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c153001-002a-4bfe-ba0c-8b2e31c4a77f None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:45.072433 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c756ca-92db-419f-a3d1-fa01c27cc362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:45.073526 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c756ca-92db-419f-a3d1-fa01c27cc362 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:45.073526 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c756ca-92db-419f-a3d1-fa01c27cc362 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:45.073727 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c756ca-92db-419f-a3d1-fa01c27cc362 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:45.101725 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:45.114720 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4bb865-5ec8-4af6-9c30-c0130a00e6ca None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:45.146314 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c756ca-92db-419f-a3d1-fa01c27cc362 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:45.147792 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c756ca-92db-419f-a3d1-fa01c27cc362 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:45.152027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08a9e55-ad1e-43a7-b771-21a3fe0e91de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:45.152862 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a9e55-ad1e-43a7-b771-21a3fe0e91de None admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:45.153350 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a9e55-ad1e-43a7-b771-21a3fe0e91de None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:45.153830 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a9e55-ad1e-43a7-b771-21a3fe0e91de None admin] PATH_INFO: `/v3/endpoints` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:45.154739 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08a9e55-ad1e-43a7-b771-21a3fe0e91de None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:45.156403 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08a9e55-ad1e-43a7-b771-21a3fe0e91de None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:45.188590 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 417/821] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:13:44 2022] POST /identity/v3/endpoints => generated 346 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:13:45.190628 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df10caa-0b3d-4953-9ab2-82f1d623eefa None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:45.191345 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df10caa-0b3d-4953-9ab2-82f1d623eefa None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:45.191970 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df10caa-0b3d-4953-9ab2-82f1d623eefa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:45.216731 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-0df10caa-0b3d-4953-9ab2-82f1d623eefa None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:45.218850 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-0df10caa-0b3d-4953-9ab2-82f1d623eefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:45.312373 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 418/822] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:13:45.323961 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:45.336588 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 405/823] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:13:44 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 24 14:13:45.344137 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df10caa-0b3d-4953-9ab2-82f1d623eefa None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:45.440529 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:45.479477 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df10caa-0b3d-4953-9ab2-82f1d623eefa None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:45.516706 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f72db1-1dce-4767-9824-0c047fc10aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:45.517600 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f72db1-1dce-4767-9824-0c047fc10aac None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:45.518020 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f72db1-1dce-4767-9824-0c047fc10aac None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:45.518705 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f72db1-1dce-4767-9824-0c047fc10aac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:45.531167 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f72db1-1dce-4767-9824-0c047fc10aac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:45.532806 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f72db1-1dce-4767-9824-0c047fc10aac None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:45.548328 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-78f72db1-1dce-4767-9824-0c047fc10aac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:45.548328 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:45.548328 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:45.548328 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:45.548328 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:45.548328 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:45.548328 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:45.548328 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:45.548328 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:45.548328 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:45.548328 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:45.548328 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:45.548328 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:45.548328 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:45.548328 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:45.548328 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:45.549502 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:45.549502 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:45.549502 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:45.549502 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:45.549502 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:45.549502 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:45.549502 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:45.549502 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:45.549502 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:45.549502 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:45.549502 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:45.549502 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:45.549502 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:45.549502 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:45.549502 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:45.549502 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:45.549502 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:45.550595 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:45.550595 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:45.550595 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:45.552421 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 419/824] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:45.558392 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c756ca-92db-419f-a3d1-fa01c27cc362 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:45.588375 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f72db1-1dce-4767-9824-0c047fc10aac None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:45.682321 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c756ca-92db-419f-a3d1-fa01c27cc362 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:45.729953 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f72db1-1dce-4767-9824-0c047fc10aac None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:45.739498 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b06340-0a5e-4fa1-9dfe-95c63ed5ee04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:45.740199 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b06340-0a5e-4fa1-9dfe-95c63ed5ee04 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:45.740378 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b06340-0a5e-4fa1-9dfe-95c63ed5ee04 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:45.740621 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b06340-0a5e-4fa1-9dfe-95c63ed5ee04 None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:45.742926 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b06340-0a5e-4fa1-9dfe-95c63ed5ee04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:45.742926 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b06340-0a5e-4fa1-9dfe-95c63ed5ee04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:45.742926 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b06340-0a5e-4fa1-9dfe-95c63ed5ee04 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:45.764966 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 406/825] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:13:45.768611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b06340-0a5e-4fa1-9dfe-95c63ed5ee04 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:45.787306 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:45.812393 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f39f2-4a4c-4fdd-a4bd-bcd70429a6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:45.813034 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f39f2-4a4c-4fdd-a4bd-bcd70429a6ff None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:45.813312 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f39f2-4a4c-4fdd-a4bd-bcd70429a6ff None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:45.813580 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f39f2-4a4c-4fdd-a4bd-bcd70429a6ff None admin] PATH_INFO: `/v3/services/network` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:45.814170 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249f39f2-4a4c-4fdd-a4bd-bcd70429a6ff None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:45.817251 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249f39f2-4a4c-4fdd-a4bd-bcd70429a6ff None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:45.839867 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-249f39f2-4a4c-4fdd-a4bd-bcd70429a6ff None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 14:13:45.839867 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:45.839867 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:13:45.839867 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:13:45.839867 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:13:45.839867 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:13:45.839867 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:13:45.839867 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:13:45.839867 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 14:13:45.839867 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:45.839867 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:13:45.839867 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:45.839867 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:45.839867 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:45.839867 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:45.839867 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:45.839867 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:45.841036 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:45.841036 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:45.841036 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:45.841036 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:45.841036 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:45.841036 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:45.841036 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:13:45.841036 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:13:45.841036 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:13:45.841036 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:13:45.841036 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:45.841036 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:45.841036 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:45.841036 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:45.841036 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:45.841036 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:45.842024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:45.842024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:45.842024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:45.842024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:45.842024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:45.842024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:45.842024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:45.842024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:45.842024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:45.842024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:45.842024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:13:45.842024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:13:45.842024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 14:13:45.842024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:45.845216 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 420/826] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:13:45 2022] GET /identity/v3/services/network => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:13:45.848759 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f39f2-4a4c-4fdd-a4bd-bcd70429a6ff None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:45.867186 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:45.931518 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b06340-0a5e-4fa1-9dfe-95c63ed5ee04 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:45.999317 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853375b-e6a6-4cbe-a083-d70618e3a588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:45.999966 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853375b-e6a6-4cbe-a083-d70618e3a588 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:46.000241 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853375b-e6a6-4cbe-a083-d70618e3a588 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:46.000511 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853375b-e6a6-4cbe-a083-d70618e3a588 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:46.013362 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b853375b-e6a6-4cbe-a083-d70618e3a588 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:46.014839 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b853375b-e6a6-4cbe-a083-d70618e3a588 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:46.036263 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-b853375b-e6a6-4cbe-a083-d70618e3a588 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:46.036263 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:46.036263 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:46.036263 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:46.036263 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:46.036263 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:46.036263 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:46.036263 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:46.036263 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:46.036263 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:46.036263 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:46.036263 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:46.036263 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:13:46.036263 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:13:46.036263 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:13:46.037193 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:13:46.037193 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:46.037193 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:46.037193 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:46.037193 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:46.037193 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:46.037193 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:46.037193 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:46.037193 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:46.037193 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:46.037193 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:46.037193 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:46.037193 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:46.037193 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:46.037193 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:46.037193 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:46.037193 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:46.038058 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:13:46.038058 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:13:46.038058 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:13:46.038058 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:13:46.038058 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:13:46.038058 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:13:46.038058 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:46.038058 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:46.038907 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 407/827] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:13:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:46.046059 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853375b-e6a6-4cbe-a083-d70618e3a588 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:46.052234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f39f2-4a4c-4fdd-a4bd-bcd70429a6ff None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:46.109513 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3243cc6f-b70e-4fac-9bc4-88da9351645c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:46.110049 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243cc6f-b70e-4fac-9bc4-88da9351645c None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:46.110324 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243cc6f-b70e-4fac-9bc4-88da9351645c None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:46.110594 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243cc6f-b70e-4fac-9bc4-88da9351645c None admin] PATH_INFO: `/v3/services` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:46.111259 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3243cc6f-b70e-4fac-9bc4-88da9351645c None admin] RBAC: Adding query filter params (name=network) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:46.111558 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3243cc6f-b70e-4fac-9bc4-88da9351645c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:46.113035 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3243cc6f-b70e-4fac-9bc4-88da9351645c None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:46.118078 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 421/828] 10.222.0.52 () {56 vars in 1112 bytes} [Sat Sep 24 14:13:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:13:46.124597 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3243cc6f-b70e-4fac-9bc4-88da9351645c None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:46.220102 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:46.244952 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853375b-e6a6-4cbe-a083-d70618e3a588 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:46.269355 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:46.289992 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3243cc6f-b70e-4fac-9bc4-88da9351645c None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:46.301561 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb623c6-4503-495b-809c-b07d51307eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:46.302185 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb623c6-4503-495b-809c-b07d51307eaf None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:46.302459 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb623c6-4503-495b-809c-b07d51307eaf None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:46.302726 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb623c6-4503-495b-809c-b07d51307eaf None admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:46.303464 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb623c6-4503-495b-809c-b07d51307eaf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:46.303762 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb623c6-4503-495b-809c-b07d51307eaf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:46.306049 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb623c6-4503-495b-809c-b07d51307eaf None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:46.321017 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 408/829] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 14:13:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:46.327330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb623c6-4503-495b-809c-b07d51307eaf None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:46.358027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302db201-ca8c-46ec-9230-cfa904b45d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:46.358647 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302db201-ca8c-46ec-9230-cfa904b45d82 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:46.358940 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302db201-ca8c-46ec-9230-cfa904b45d82 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:46.359206 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302db201-ca8c-46ec-9230-cfa904b45d82 None admin] PATH_INFO: `/v3/services` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:46.360191 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302db201-ca8c-46ec-9230-cfa904b45d82 None admin] RBAC: Adding query filter params (type=network) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:46.360433 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302db201-ca8c-46ec-9230-cfa904b45d82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:46.361801 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302db201-ca8c-46ec-9230-cfa904b45d82 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:46.374099 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 422/830] 10.222.0.52 () {56 vars in 1112 bytes} [Sat Sep 24 14:13:46 2022] GET /identity/v3/services?type=network => generated 350 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:46.381067 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302db201-ca8c-46ec-9230-cfa904b45d82 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:46.486570 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb623c6-4503-495b-809c-b07d51307eaf None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:46.525078 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302db201-ca8c-46ec-9230-cfa904b45d82 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:46.533868 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff3182-0b7f-4eef-ba8f-85a03efd8d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:46.535022 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff3182-0b7f-4eef-ba8f-85a03efd8d07 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:46.535299 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff3182-0b7f-4eef-ba8f-85a03efd8d07 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:46.535567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff3182-0b7f-4eef-ba8f-85a03efd8d07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:46.536324 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ff3182-0b7f-4eef-ba8f-85a03efd8d07 None admin] RBAC: Adding query filter params (role.id=d00b78c32dbf4cc7925a55bb621d8fcf, scope.project.id=36a3ae7bab25471fb989136150b6438c, user.id=79eaaae97aac4282a613939e5b768376) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:46.537052 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ff3182-0b7f-4eef-ba8f-85a03efd8d07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:46.538941 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ff3182-0b7f-4eef-ba8f-85a03efd8d07 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:46.545306 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:46.547534 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 409/831] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:13:46 2022] GET /identity/v3/role_assignments?user.id=79eaaae97aac4282a613939e5b768376&scope.project.id=36a3ae7bab25471fb989136150b6438c&role.id=d00b78c32dbf4cc7925a55bb621d8fcf => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:13:46.548997 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32aedf-7d83-45e0-abbc-84bd6b1d0bad None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:46.549838 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32aedf-7d83-45e0-abbc-84bd6b1d0bad None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:46.550264 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32aedf-7d83-45e0-abbc-84bd6b1d0bad None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:46.552302 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 410/832] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:46.558680 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1c0da-e83b-4839-99f6-03e739f08eef None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:46.559347 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1c0da-e83b-4839-99f6-03e739f08eef None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:46.559347 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1c0da-e83b-4839-99f6-03e739f08eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:46.570183 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f0bad-f198-48bd-b9a7-80df966d2a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:46.570947 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f0bad-f198-48bd-b9a7-80df966d2a14 None admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:46.571323 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f0bad-f198-48bd-b9a7-80df966d2a14 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:46.572340 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f0bad-f198-48bd-b9a7-80df966d2a14 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:46.573154 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1f0bad-f198-48bd-b9a7-80df966d2a14 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:46.574736 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1f0bad-f198-48bd-b9a7-80df966d2a14 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:46.582764 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-21c1c0da-e83b-4839-99f6-03e739f08eef None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:46.584447 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-21c1c0da-e83b-4839-99f6-03e739f08eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:46.599662 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 423/833] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:13:46 2022] POST /identity/v3/endpoints => generated 337 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:13:46.616604 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:46.671801 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 411/834] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:46 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:13:47.014268 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f0bad-f198-48bd-b9a7-80df966d2a14 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:47.071103 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15582a4b-c7fb-4b0b-a07b-472828ee3768 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:47.071497 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15582a4b-c7fb-4b0b-a07b-472828ee3768 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:47.072241 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15582a4b-c7fb-4b0b-a07b-472828ee3768 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:47.073824 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 412/835] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:47.078881 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324845b-2ec5-4ea2-8c11-63267c0a7338 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:47.079169 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324845b-2ec5-4ea2-8c11-63267c0a7338 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:47.079446 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324845b-2ec5-4ea2-8c11-63267c0a7338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:47.098089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-2324845b-2ec5-4ea2-8c11-63267c0a7338 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:47.099596 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-2324845b-2ec5-4ea2-8c11-63267c0a7338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:47.122416 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f0bad-f198-48bd-b9a7-80df966d2a14 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:47.160040 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72413ff-073d-451e-aa81-02cc4efa27a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:47.160669 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72413ff-073d-451e-aa81-02cc4efa27a8 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:47.160945 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72413ff-073d-451e-aa81-02cc4efa27a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:47.161211 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72413ff-073d-451e-aa81-02cc4efa27a8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:47.171200 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 413/836] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:47 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:13:47.178580 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72413ff-073d-451e-aa81-02cc4efa27a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:13:47.178580 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-f72413ff-073d-451e-aa81-02cc4efa27a8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:13:47.178580 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:47.178580 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:47.178580 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:47.178580 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:47.178580 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:47.178580 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:47.178580 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:47.178580 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:47.178580 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:47.178580 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:47.178580 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:47.178580 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:13:47.179530 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:13:47.179530 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:13:47.179530 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:13:47.179530 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:13:47.179530 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:13:47.179530 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:13:47.179530 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:13:47.179530 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:47.179530 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:47.179530 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:47.179530 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:47.179530 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:47.179530 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:47.179530 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:47.179530 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:47.179530 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:47.179530 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:47.180602 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:47.180602 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:47.180602 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:47.180602 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:47.180602 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:47.180602 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:47.180602 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:13:47.180602 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:13:47.180602 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:13:47.180602 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:13:47.180602 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:13:47.180602 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:13:47.180602 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:13:47.180602 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:47.180602 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 424/837] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 14:13:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:47.187797 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324845b-2ec5-4ea2-8c11-63267c0a7338 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:47.280576 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:47.292679 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324845b-2ec5-4ea2-8c11-63267c0a7338 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:47.327322 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad6bb21-a395-4f42-9dcd-a27873cd8111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:47.327928 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad6bb21-a395-4f42-9dcd-a27873cd8111 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:47.328204 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad6bb21-a395-4f42-9dcd-a27873cd8111 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:47.328473 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad6bb21-a395-4f42-9dcd-a27873cd8111 None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:47.329137 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad6bb21-a395-4f42-9dcd-a27873cd8111 None admin] RBAC: Adding query filter params (name=service) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:47.329436 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad6bb21-a395-4f42-9dcd-a27873cd8111 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:47.330788 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad6bb21-a395-4f42-9dcd-a27873cd8111 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:47.339867 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 414/838] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 14:13:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:47.345498 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72413ff-073d-451e-aa81-02cc4efa27a8 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:47.380285 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:47.450109 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad6bb21-a395-4f42-9dcd-a27873cd8111 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:47.454512 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72413ff-073d-451e-aa81-02cc4efa27a8 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:47.491380 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27919fbb-8422-4277-8eea-c27d39e0680a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:47.492144 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27919fbb-8422-4277-8eea-c27d39e0680a None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:47.492437 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27919fbb-8422-4277-8eea-c27d39e0680a None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:47.492742 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27919fbb-8422-4277-8eea-c27d39e0680a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:47.505870 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27919fbb-8422-4277-8eea-c27d39e0680a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:47.507352 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27919fbb-8422-4277-8eea-c27d39e0680a None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:47.530021 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-27919fbb-8422-4277-8eea-c27d39e0680a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:47.530021 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:47.530021 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:47.530021 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:47.530021 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:47.530021 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:47.530021 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:47.530021 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:47.530021 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:47.530021 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:47.530021 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:47.530021 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:47.530021 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:47.530021 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:47.530021 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:47.530021 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:47.531108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:47.531108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:47.531108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:47.531108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:47.531108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:47.531108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:47.531108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:47.531108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:47.531108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:47.531108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:47.531108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:47.531108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:47.531108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:47.531108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:47.531108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:47.531108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:47.531108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:47.532029 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:47.532029 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:47.532029 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:47.532745 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 425/839] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:47.538029 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27919fbb-8422-4277-8eea-c27d39e0680a None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:47.579218 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad6bb21-a395-4f42-9dcd-a27873cd8111 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:47.622111 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a857b2a-8a44-4dc2-81ba-f611b8bb5388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:47.622927 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a857b2a-8a44-4dc2-81ba-f611b8bb5388 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:47.623278 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a857b2a-8a44-4dc2-81ba-f611b8bb5388 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:47.623620 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a857b2a-8a44-4dc2-81ba-f611b8bb5388 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:47.635213 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a857b2a-8a44-4dc2-81ba-f611b8bb5388 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:13:47.639795 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:47.641029 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-9a857b2a-8a44-4dc2-81ba-f611b8bb5388 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:13:47.641029 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:47.641029 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:47.641029 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:47.641029 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:47.641029 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:47.641029 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:47.641029 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:47.641029 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:47.641029 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:47.641029 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:47.641029 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:47.641029 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:13:47.641029 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:13:47.641029 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:13:47.641029 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:13:47.641951 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:13:47.641951 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:13:47.641951 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:13:47.641951 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:13:47.641951 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:47.641951 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:47.641951 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:47.641951 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:47.641951 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:47.641951 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:47.641951 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:47.641951 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:47.641951 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:47.641951 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:47.641951 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:47.641951 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:47.647257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:47.647257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:47.647257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:47.647257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:47.647257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:13:47.647257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:13:47.647257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:13:47.647257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:13:47.647257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:13:47.647257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:13:47.647257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:13:47.647257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:47.647257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 415/840] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 14:13:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:13:47.651782 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a857b2a-8a44-4dc2-81ba-f611b8bb5388 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:47.664872 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27919fbb-8422-4277-8eea-c27d39e0680a None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:47.671837 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:47.702539 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b3e554-daa2-41bc-984e-01ebf95ab435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:47.703504 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b3e554-daa2-41bc-984e-01ebf95ab435 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:47.703885 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b3e554-daa2-41bc-984e-01ebf95ab435 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:47.704255 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b3e554-daa2-41bc-984e-01ebf95ab435 None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:47.705517 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b3e554-daa2-41bc-984e-01ebf95ab435 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:47.705517 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b3e554-daa2-41bc-984e-01ebf95ab435 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:47.707250 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b3e554-daa2-41bc-984e-01ebf95ab435 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:47.730133 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 426/841] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:13:47.731367 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b3e554-daa2-41bc-984e-01ebf95ab435 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:47.791204 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a857b2a-8a44-4dc2-81ba-f611b8bb5388 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:47.837360 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c4f8f3-740e-4a79-9f2b-25c9cc5b9a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:47.838218 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4f8f3-740e-4a79-9f2b-25c9cc5b9a32 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:47.838645 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4f8f3-740e-4a79-9f2b-25c9cc5b9a32 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:47.839029 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b3e554-daa2-41bc-984e-01ebf95ab435 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:47.839277 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4f8f3-740e-4a79-9f2b-25c9cc5b9a32 None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:47.840675 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c4f8f3-740e-4a79-9f2b-25c9cc5b9a32 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:47.841278 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c4f8f3-740e-4a79-9f2b-25c9cc5b9a32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:47.843670 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c4f8f3-740e-4a79-9f2b-25c9cc5b9a32 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:47.854074 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 416/842] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:13:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:47.855560 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7d4ae-0f31-4bca-8e32-9775633551b2 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:47.856000 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7d4ae-0f31-4bca-8e32-9775633551b2 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:47.856271 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7d4ae-0f31-4bca-8e32-9775633551b2 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:47.857578 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 417/843] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:47.863878 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140e6ad-e633-44cf-ae7c-c481a2c7f331 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:47.864264 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140e6ad-e633-44cf-ae7c-c481a2c7f331 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:47.864545 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140e6ad-e633-44cf-ae7c-c481a2c7f331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:47.877811 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73740a4f-4025-433e-a741-3d2e18e02b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:47.878436 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73740a4f-4025-433e-a741-3d2e18e02b47 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:47.878731 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73740a4f-4025-433e-a741-3d2e18e02b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:47.879037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73740a4f-4025-433e-a741-3d2e18e02b47 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:47.883015 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-d140e6ad-e633-44cf-ae7c-c481a2c7f331 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:47.884769 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-d140e6ad-e633-44cf-ae7c-c481a2c7f331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:47.886836 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73740a4f-4025-433e-a741-3d2e18e02b47 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:47.888392 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73740a4f-4025-433e-a741-3d2e18e02b47 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:47.902138 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-73740a4f-4025-433e-a741-3d2e18e02b47 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 14:13:47.902138 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:47.902138 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:47.902138 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:47.902138 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:47.902138 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:47.902138 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:47.902138 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:47.902138 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:47.902138 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:47.902138 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:47.902138 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:47.902138 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:13:47.902138 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:13:47.902138 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:13:47.902138 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:13:47.903245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:47.903245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:47.903245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:13:47.903245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:47.903245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:13:47.903245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:47.903245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:47.903245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:47.903245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:47.903245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:47.903245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:47.903245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:47.903245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:47.903245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:47.903245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:47.903245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:47.903245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:47.903245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:47.904735 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:47.904735 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:47.904735 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:13:47.904735 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:13:47.904735 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:13:47.904735 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:13:47.904735 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:13:47.904735 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:13:47.904735 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 14:13:47.904735 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:47.904735 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 427/844] 10.222.0.52 () {56 vars in 1117 bytes} [Sat Sep 24 14:13:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:47.908333 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73740a4f-4025-433e-a741-3d2e18e02b47 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:47.912564 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:47.958692 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 418/845] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:47 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:13:47.959901 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140e6ad-e633-44cf-ae7c-c481a2c7f331 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:47.983995 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:48.016281 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73740a4f-4025-433e-a741-3d2e18e02b47 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:48.051950 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f32ce9d-6308-474d-9d23-4908178a4007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:48.052502 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32ce9d-6308-474d-9d23-4908178a4007 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:48.052773 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32ce9d-6308-474d-9d23-4908178a4007 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:48.053037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32ce9d-6308-474d-9d23-4908178a4007 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:48.063882 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f32ce9d-6308-474d-9d23-4908178a4007 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:48.065335 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f32ce9d-6308-474d-9d23-4908178a4007 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:48.080338 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-6f32ce9d-6308-474d-9d23-4908178a4007 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:48.080338 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:48.080338 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:48.080338 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:48.080338 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:48.080338 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:48.080338 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:48.080338 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:48.080338 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:48.080338 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:48.080338 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:48.080338 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:48.080338 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:48.080338 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:48.080338 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:48.080338 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:48.081294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:48.081294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:48.081294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:48.081294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:48.081294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:48.081294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:48.081294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:48.081294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:48.081294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:48.081294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:48.081294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:48.081294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:48.081294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:48.081294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:48.081294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:48.081294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:48.081294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:48.082217 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:48.082217 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:48.082217 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:48.082217 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 428/846] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:48.087881 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f32ce9d-6308-474d-9d23-4908178a4007 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:48.090151 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140e6ad-e633-44cf-ae7c-c481a2c7f331 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:48.125375 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd03d3c0-6cdc-427e-ad10-86e006ed6f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:48.126035 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03d3c0-6cdc-427e-ad10-86e006ed6f10 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:48.126308 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03d3c0-6cdc-427e-ad10-86e006ed6f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:48.126575 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03d3c0-6cdc-427e-ad10-86e006ed6f10 None admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:48.127372 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd03d3c0-6cdc-427e-ad10-86e006ed6f10 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:48.127670 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd03d3c0-6cdc-427e-ad10-86e006ed6f10 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:48.129080 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd03d3c0-6cdc-427e-ad10-86e006ed6f10 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:48.152055 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 419/847] 10.222.0.52 () {56 vars in 1149 bytes} [Sat Sep 24 14:13:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:48.162835 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd03d3c0-6cdc-427e-ad10-86e006ed6f10 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:48.169750 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:48.224528 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f32ce9d-6308-474d-9d23-4908178a4007 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:48.236182 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:48.262519 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b2108-e666-4e7b-9270-7745c3e06676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:48.263162 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b2108-e666-4e7b-9270-7745c3e06676 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:48.263441 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b2108-e666-4e7b-9270-7745c3e06676 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:48.263715 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b2108-e666-4e7b-9270-7745c3e06676 None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:48.264417 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411b2108-e666-4e7b-9270-7745c3e06676 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:48.264718 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411b2108-e666-4e7b-9270-7745c3e06676 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:48.266112 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411b2108-e666-4e7b-9270-7745c3e06676 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:48.277343 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd03d3c0-6cdc-427e-ad10-86e006ed6f10 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:48.316065 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ecad4-0a99-400f-9826-ee120749a014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:48.316607 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ecad4-0a99-400f-9826-ee120749a014 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:48.316888 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ecad4-0a99-400f-9826-ee120749a014 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:48.317160 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ecad4-0a99-400f-9826-ee120749a014 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:48.390437 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1ecad4-0a99-400f-9826-ee120749a014 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:48.391923 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1ecad4-0a99-400f-9826-ee120749a014 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:48.394791 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 429/848] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:13:48.395666 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b2108-e666-4e7b-9270-7745c3e06676 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:48.461782 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:48.499432 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b2108-e666-4e7b-9270-7745c3e06676 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:48.509296 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 420/849] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:13:48 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 24 14:13:48.509296 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ecad4-0a99-400f-9826-ee120749a014 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:48.520047 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:48.539512 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8779b9ec-54de-429b-8675-938e5e313bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:48.540198 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779b9ec-54de-429b-8675-938e5e313bfc None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:48.540966 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779b9ec-54de-429b-8675-938e5e313bfc None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:48.541223 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779b9ec-54de-429b-8675-938e5e313bfc None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:48.541825 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8779b9ec-54de-429b-8675-938e5e313bfc None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:48.543183 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8779b9ec-54de-429b-8675-938e5e313bfc None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:48.552108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-8779b9ec-54de-429b-8675-938e5e313bfc None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 14:13:48.552108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:48.552108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:13:48.552108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:13:48.552108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:13:48.552108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:13:48.552108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:13:48.552108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:13:48.552108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 14:13:48.552108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:48.552108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:13:48.552108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:48.552108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:48.552108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:48.552108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:48.552108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:48.552108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:48.553118 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:48.553118 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:48.553118 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:48.553118 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:48.553118 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:48.553118 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:48.553118 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:13:48.553118 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:13:48.553118 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:13:48.553118 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:13:48.553118 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:48.553118 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:48.553118 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:48.553118 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:48.553118 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:48.553118 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:48.553943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:48.553943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:48.553943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:48.553943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:48.553943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:48.553943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:48.553943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:48.553943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:48.553943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:48.553943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:48.553943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:13:48.553943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:13:48.553943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 14:13:48.553943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:48.553943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 430/850] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 14:13:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:48.554725 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8779b9ec-54de-429b-8675-938e5e313bfc None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:48.616604 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ecad4-0a99-400f-9826-ee120749a014 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:48.651972 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ddac4-f632-430f-b470-80480a353602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:48.652599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ddac4-f632-430f-b470-80480a353602 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:48.652879 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ddac4-f632-430f-b470-80480a353602 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:48.653154 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ddac4-f632-430f-b470-80480a353602 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:48.660334 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ddac4-f632-430f-b470-80480a353602 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:48.661753 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ddac4-f632-430f-b470-80480a353602 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:48.663661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8779b9ec-54de-429b-8675-938e5e313bfc None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:48.673641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-ba5ddac4-f632-430f-b470-80480a353602 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 24 14:13:48.673641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:48.673641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:48.673641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:48.673641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:48.673641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:48.673641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:48.673641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:48.673641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:48.673641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:48.673641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:48.673641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:48.673641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:13:48.673641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:13:48.673641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:13:48.673641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:13:48.674805 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:48.674805 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:48.674805 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:13:48.674805 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:48.674805 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:13:48.674805 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:48.674805 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:48.674805 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:48.674805 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:48.674805 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:48.674805 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:48.674805 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:48.674805 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:48.674805 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:48.674805 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:48.674805 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:48.674805 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:48.674805 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:48.675757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:48.675757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:48.675757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:13:48.675757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:13:48.675757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:13:48.675757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:13:48.675757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:13:48.675757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:13:48.675757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 24 14:13:48.675757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:48.675757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 421/851] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 14:13:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:48.676338 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ddac4-f632-430f-b470-80480a353602 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:48.703160 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e41856-2c54-4966-a3a1-2f868ced5150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:48.703366 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:48.703786 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e41856-2c54-4966-a3a1-2f868ced5150 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:48.704060 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e41856-2c54-4966-a3a1-2f868ced5150 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:48.704329 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e41856-2c54-4966-a3a1-2f868ced5150 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:48.716491 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e41856-2c54-4966-a3a1-2f868ced5150 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:48.717939 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e41856-2c54-4966-a3a1-2f868ced5150 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:48.732676 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-28e41856-2c54-4966-a3a1-2f868ced5150 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:48.732676 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:48.732676 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:48.732676 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:48.732676 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:48.732676 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:48.732676 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:48.732676 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:48.732676 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:48.732676 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:48.732676 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:48.732676 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:48.732676 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:48.732676 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:48.732676 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:48.732676 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:48.733641 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:48.733641 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:48.733641 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:48.733641 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:48.733641 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:48.733641 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:48.733641 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:48.733641 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:48.733641 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:48.733641 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:48.733641 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:48.733641 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:48.733641 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:48.733641 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:48.733641 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:48.733641 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:48.733641 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:48.734507 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:48.734507 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:48.734507 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:48.738352 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 431/852] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:48.739208 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e41856-2c54-4966-a3a1-2f868ced5150 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:48.769759 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:48.789328 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ddac4-f632-430f-b470-80480a353602 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:48.824253 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac9075b-9c5e-4cf6-9f92-b144d7ec3e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:48.824852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9075b-9c5e-4cf6-9f92-b144d7ec3e4e None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:48.825124 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9075b-9c5e-4cf6-9f92-b144d7ec3e4e None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:48.825391 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9075b-9c5e-4cf6-9f92-b144d7ec3e4e None admin] PATH_INFO: `/v3/services` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:48.826006 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac9075b-9c5e-4cf6-9f92-b144d7ec3e4e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:48.826303 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac9075b-9c5e-4cf6-9f92-b144d7ec3e4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:48.827693 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac9075b-9c5e-4cf6-9f92-b144d7ec3e4e None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:48.834036 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 422/853] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 14:13:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:48.834036 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac9075b-9c5e-4cf6-9f92-b144d7ec3e4e None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:48.852213 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e41856-2c54-4966-a3a1-2f868ced5150 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:48.887776 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8555f8d5-fb3e-4bdd-b911-32b4fdf47322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:48.888387 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555f8d5-fb3e-4bdd-b911-32b4fdf47322 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:48.888669 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555f8d5-fb3e-4bdd-b911-32b4fdf47322 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:48.888943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555f8d5-fb3e-4bdd-b911-32b4fdf47322 None admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:48.889671 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8555f8d5-fb3e-4bdd-b911-32b4fdf47322 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:48.890062 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8555f8d5-fb3e-4bdd-b911-32b4fdf47322 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:48.891535 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8555f8d5-fb3e-4bdd-b911-32b4fdf47322 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:48.914908 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 432/854] 10.222.0.52 () {56 vars in 1135 bytes} [Sat Sep 24 14:13:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:48.914908 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8555f8d5-fb3e-4bdd-b911-32b4fdf47322 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:48.927631 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:48.947828 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac9075b-9c5e-4cf6-9f92-b144d7ec3e4e None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:48.986889 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e4e86-64b8-4f00-9ed9-3e9f11d4e3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:48.987504 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e4e86-64b8-4f00-9ed9-3e9f11d4e3e5 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:48.987780 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e4e86-64b8-4f00-9ed9-3e9f11d4e3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:48.988050 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e4e86-64b8-4f00-9ed9-3e9f11d4e3e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:48.988706 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2e4e86-64b8-4f00-9ed9-3e9f11d4e3e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:48.989012 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2e4e86-64b8-4f00-9ed9-3e9f11d4e3e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:48.990407 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2e4e86-64b8-4f00-9ed9-3e9f11d4e3e5 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:49.002952 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 423/855] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:49.003808 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e4e86-64b8-4f00-9ed9-3e9f11d4e3e5 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:49.019907 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:49.033533 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8555f8d5-fb3e-4bdd-b911-32b4fdf47322 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:49.069438 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c7650-4712-487a-a5df-59e7bb9e4a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:49.070039 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c7650-4712-487a-a5df-59e7bb9e4a1a None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:49.070308 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c7650-4712-487a-a5df-59e7bb9e4a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:49.070579 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c7650-4712-487a-a5df-59e7bb9e4a1a None admin] PATH_INFO: `/v3/services` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:49.071229 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8c7650-4712-487a-a5df-59e7bb9e4a1a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:49.072055 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8c7650-4712-487a-a5df-59e7bb9e4a1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:49.073420 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8c7650-4712-487a-a5df-59e7bb9e4a1a None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:49.077752 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 433/856] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 14:13:48 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:49.078765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c7650-4712-487a-a5df-59e7bb9e4a1a None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:49.127992 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e4e86-64b8-4f00-9ed9-3e9f11d4e3e5 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:49.169920 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edfaf48-d12a-49bf-99bc-2f7dc6348752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:49.170913 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edfaf48-d12a-49bf-99bc-2f7dc6348752 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:49.171363 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edfaf48-d12a-49bf-99bc-2f7dc6348752 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:49.171644 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edfaf48-d12a-49bf-99bc-2f7dc6348752 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:49.189378 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:49.191934 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c7650-4712-487a-a5df-59e7bb9e4a1a None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:49.248159 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c3898e-0cbc-4886-9898-7608d2ed3a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:49.249173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3898e-0cbc-4886-9898-7608d2ed3a54 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:49.249549 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edfaf48-d12a-49bf-99bc-2f7dc6348752 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:49.249709 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3898e-0cbc-4886-9898-7608d2ed3a54 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:49.250000 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3898e-0cbc-4886-9898-7608d2ed3a54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:49.251041 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edfaf48-d12a-49bf-99bc-2f7dc6348752 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:49.260204 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c3898e-0cbc-4886-9898-7608d2ed3a54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:49.262505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c3898e-0cbc-4886-9898-7608d2ed3a54 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:49.275328 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-d5c3898e-0cbc-4886-9898-7608d2ed3a54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:49.275328 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:49.275328 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:49.275328 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:49.275328 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:49.275328 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:49.275328 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:49.275328 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:49.275328 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:49.275328 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:49.275328 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:49.275328 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:49.275328 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:13:49.275328 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:13:49.275328 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:13:49.276405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:13:49.276405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:49.276405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:49.276405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:49.276405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:49.276405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:49.276405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:49.276405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:49.276405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:49.276405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:49.276405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:49.276405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:49.276405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:49.276405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:49.276405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:49.276405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:49.276405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:49.277508 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:13:49.277508 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:13:49.277508 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:13:49.277508 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:13:49.277508 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:13:49.277508 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:13:49.277508 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:49.277508 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:49.277508 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 434/857] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:13:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:49.278104 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646d32f-3ae1-4284-87ae-d240ccb09629 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:49.278184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646d32f-3ae1-4284-87ae-d240ccb09629 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:49.278427 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646d32f-3ae1-4284-87ae-d240ccb09629 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:49.284164 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 435/858] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:49.284937 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a646d32f-3ae1-4284-87ae-d240ccb09629 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:49.302463 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:49.358783 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 424/859] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:13:49 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 14:13:49.360926 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab33bd-a3bd-4384-a689-7f14d6c03343 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:49.361211 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab33bd-a3bd-4384-a689-7f14d6c03343 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:49.361483 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab33bd-a3bd-4384-a689-7f14d6c03343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:49.381761 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-dfab33bd-a3bd-4384-a689-7f14d6c03343 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:49.382963 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-dfab33bd-a3bd-4384-a689-7f14d6c03343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:49.391534 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a646d32f-3ae1-4284-87ae-d240ccb09629 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:49.428240 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff86fc-667c-46a4-8f1c-898610c76af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:49.429081 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff86fc-667c-46a4-8f1c-898610c76af5 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:49.429354 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff86fc-667c-46a4-8f1c-898610c76af5 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:49.429624 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff86fc-667c-46a4-8f1c-898610c76af5 None admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:49.430309 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ff86fc-667c-46a4-8f1c-898610c76af5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:49.430611 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ff86fc-667c-46a4-8f1c-898610c76af5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:49.431746 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:49.432124 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ff86fc-667c-46a4-8f1c-898610c76af5 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:49.444878 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 436/860] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 14:13:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:49.448000 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff86fc-667c-46a4-8f1c-898610c76af5 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:49.466835 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 425/861] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:49 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:13:49.466835 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab33bd-a3bd-4384-a689-7f14d6c03343 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:49.542062 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:49.571694 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff86fc-667c-46a4-8f1c-898610c76af5 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:49.609920 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab33bd-a3bd-4384-a689-7f14d6c03343 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:49.610700 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27ab051-e2f7-44a9-bd3c-7213e3e1e05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:49.611456 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ab051-e2f7-44a9-bd3c-7213e3e1e05f None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:49.611830 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ab051-e2f7-44a9-bd3c-7213e3e1e05f None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:49.612199 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ab051-e2f7-44a9-bd3c-7213e3e1e05f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:49.623113 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27ab051-e2f7-44a9-bd3c-7213e3e1e05f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:49.624714 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27ab051-e2f7-44a9-bd3c-7213e3e1e05f None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:49.640169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-e27ab051-e2f7-44a9-bd3c-7213e3e1e05f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:49.640169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:49.640169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:49.640169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:49.640169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:49.640169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:49.640169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:49.640169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:49.640169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:49.640169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:49.640169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:49.640169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:49.640169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:49.640169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:49.640169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:49.640169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:49.641205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:49.641205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:49.641205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:49.641205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:49.641205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:49.641205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:49.641205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:49.641205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:49.641205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:49.641205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:49.641205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:49.641205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:49.641205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:49.641205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:49.641205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:49.641205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:49.641205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:49.642083 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:49.642083 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:49.642083 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:49.646066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 437/862] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:49.646938 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27ab051-e2f7-44a9-bd3c-7213e3e1e05f None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:49.652215 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85bafe-0d8b-49f2-bdeb-0a511ecbba1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:49.652831 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85bafe-0d8b-49f2-bdeb-0a511ecbba1c None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:49.653114 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85bafe-0d8b-49f2-bdeb-0a511ecbba1c None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:49.658833 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85bafe-0d8b-49f2-bdeb-0a511ecbba1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:49.658833 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85bafe-0d8b-49f2-bdeb-0a511ecbba1c None admin] RBAC: Adding query filter params (role.id=be958e9954f04268a9f012c775fc17e5, scope.project.id=36a3ae7bab25471fb989136150b6438c, user.id=82328b7fd50649a485c6eaf27c5597c9) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:49.658833 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85bafe-0d8b-49f2-bdeb-0a511ecbba1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:49.658833 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85bafe-0d8b-49f2-bdeb-0a511ecbba1c None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:49.664683 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 426/863] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:13:49 2022] GET /identity/v3/role_assignments?user.id=82328b7fd50649a485c6eaf27c5597c9&scope.project.id=36a3ae7bab25471fb989136150b6438c&role.id=be958e9954f04268a9f012c775fc17e5 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:49.767571 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85bafe-0d8b-49f2-bdeb-0a511ecbba1c None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:49.768857 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27ab051-e2f7-44a9-bd3c-7213e3e1e05f None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:49.798201 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:49.809411 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408db06-435e-40de-acc7-a949081afc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:49.810028 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408db06-435e-40de-acc7-a949081afc2a None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:49.810302 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408db06-435e-40de-acc7-a949081afc2a None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:49.810566 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408db06-435e-40de-acc7-a949081afc2a None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:49.811244 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7408db06-435e-40de-acc7-a949081afc2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:49.811546 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7408db06-435e-40de-acc7-a949081afc2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:49.812959 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7408db06-435e-40de-acc7-a949081afc2a None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:49.828806 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:49.829837 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 438/864] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:49.836102 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408db06-435e-40de-acc7-a949081afc2a None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:49.910424 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85bafe-0d8b-49f2-bdeb-0a511ecbba1c None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:49.955186 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408db06-435e-40de-acc7-a949081afc2a None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:49.957271 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c4827e-843c-41a6-88e1-e97f2684e860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:49.957933 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c4827e-843c-41a6-88e1-e97f2684e860 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:49.958212 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c4827e-843c-41a6-88e1-e97f2684e860 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:49.958480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c4827e-843c-41a6-88e1-e97f2684e860 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:49.959117 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c4827e-843c-41a6-88e1-e97f2684e860 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:49.961197 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c4827e-843c-41a6-88e1-e97f2684e860 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:49.975038 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-65c4827e-843c-41a6-88e1-e97f2684e860 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 14:13:49.975038 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:49.975038 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:13:49.975038 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:13:49.975038 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:13:49.975038 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:13:49.975038 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:13:49.975038 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:13:49.975038 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 14:13:49.975038 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:49.975038 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:13:49.975038 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:49.975038 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:49.975038 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:49.975038 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:49.975038 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:49.976046 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:49.976046 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:49.976046 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:49.976046 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:49.976046 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:49.976046 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:49.976046 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:49.976046 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:13:49.976046 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:13:49.976046 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:13:49.976046 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:13:49.976046 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:49.976046 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:49.976046 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:49.976046 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:49.976046 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:49.976046 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:49.977072 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:49.977072 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:49.977072 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:49.977072 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:49.977072 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:49.977072 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:49.977072 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:49.977072 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:49.977072 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:49.977072 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:49.977072 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:13:49.977072 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:13:49.977072 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 14:13:49.977072 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:49.977072 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 427/865] 10.222.0.52 () {56 vars in 1132 bytes} [Sat Sep 24 14:13:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:49.983274 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c4827e-843c-41a6-88e1-e97f2684e860 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:50.001446 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d5eb02-7b4c-4123-8997-5dbe63922f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:50.002585 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d5eb02-7b4c-4123-8997-5dbe63922f0f None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:50.003424 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d5eb02-7b4c-4123-8997-5dbe63922f0f None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:50.004124 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d5eb02-7b4c-4123-8997-5dbe63922f0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:50.013065 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d5eb02-7b4c-4123-8997-5dbe63922f0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:50.014991 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d5eb02-7b4c-4123-8997-5dbe63922f0f None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:50.030957 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-d2d5eb02-7b4c-4123-8997-5dbe63922f0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:50.030957 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:50.030957 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:50.030957 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:50.030957 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:50.030957 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:50.030957 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:50.030957 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:50.030957 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:50.030957 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:50.030957 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:50.030957 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:50.030957 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:13:50.030957 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:13:50.030957 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:13:50.032292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:13:50.032292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:50.032292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:50.032292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:50.032292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:50.032292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:50.032292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:50.032292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:50.032292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:50.032292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:50.032292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:50.032292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:50.032292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:50.032292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:50.032292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:50.032292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:50.032292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:50.033465 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:13:50.033465 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:13:50.033465 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:13:50.033465 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:13:50.033465 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:13:50.033465 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:13:50.033465 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:50.033465 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:50.033465 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 439/866] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:13:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:13:50.037463 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d5eb02-7b4c-4123-8997-5dbe63922f0f None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:50.100633 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:50.119616 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c4827e-843c-41a6-88e1-e97f2684e860 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:50.122497 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:50.162562 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d5eb02-7b4c-4123-8997-5dbe63922f0f None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:50.163000 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430d2a01-e083-4fa4-90d6-eafee1bf95ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:50.163865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d2a01-e083-4fa4-90d6-eafee1bf95ea None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:50.164401 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d2a01-e083-4fa4-90d6-eafee1bf95ea None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:50.164954 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d2a01-e083-4fa4-90d6-eafee1bf95ea None admin] PATH_INFO: `/v3/services` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:50.166223 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430d2a01-e083-4fa4-90d6-eafee1bf95ea None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:50.166529 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430d2a01-e083-4fa4-90d6-eafee1bf95ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:50.167925 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430d2a01-e083-4fa4-90d6-eafee1bf95ea None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:50.177655 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 428/867] 10.222.0.52 () {56 vars in 1126 bytes} [Sat Sep 24 14:13:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:50.179783 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430d2a01-e083-4fa4-90d6-eafee1bf95ea None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:50.205544 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db8ebe4-470a-4a92-b324-f0525b8d6052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:50.206328 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db8ebe4-470a-4a92-b324-f0525b8d6052 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:50.206707 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db8ebe4-470a-4a92-b324-f0525b8d6052 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:50.207119 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db8ebe4-470a-4a92-b324-f0525b8d6052 None admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:50.208041 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db8ebe4-470a-4a92-b324-f0525b8d6052 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:50.208445 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db8ebe4-470a-4a92-b324-f0525b8d6052 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:50.210130 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db8ebe4-470a-4a92-b324-f0525b8d6052 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:50.231380 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 440/868] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 14:13:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:50.232730 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db8ebe4-470a-4a92-b324-f0525b8d6052 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:50.298780 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430d2a01-e083-4fa4-90d6-eafee1bf95ea None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:50.339190 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0eaa41-f081-4172-b493-980d49338c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:50.340182 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0eaa41-f081-4172-b493-980d49338c69 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:50.340678 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0eaa41-f081-4172-b493-980d49338c69 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:50.341143 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0eaa41-f081-4172-b493-980d49338c69 None admin] PATH_INFO: `/v3/services` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:50.341967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0eaa41-f081-4172-b493-980d49338c69 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:50.342463 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0eaa41-f081-4172-b493-980d49338c69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:50.344084 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0eaa41-f081-4172-b493-980d49338c69 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:50.354867 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db8ebe4-470a-4a92-b324-f0525b8d6052 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:50.355291 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 429/869] 10.222.0.52 () {56 vars in 1126 bytes} [Sat Sep 24 14:13:50 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:50.359567 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:50.395115 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe3a1a1-7415-474d-a7cd-802f594e9042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:50.395996 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe3a1a1-7415-474d-a7cd-802f594e9042 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:50.396450 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe3a1a1-7415-474d-a7cd-802f594e9042 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:50.396879 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe3a1a1-7415-474d-a7cd-802f594e9042 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:50.397767 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe3a1a1-7415-474d-a7cd-802f594e9042 None admin] RBAC: Adding query filter params (role.id=d00b78c32dbf4cc7925a55bb621d8fcf, scope.project.id=36a3ae7bab25471fb989136150b6438c, user.id=879f296733114f6f844a98aaefe35e92) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:50.398224 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe3a1a1-7415-474d-a7cd-802f594e9042 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:50.399943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe3a1a1-7415-474d-a7cd-802f594e9042 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:50.407454 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 441/870] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:13:50 2022] GET /identity/v3/role_assignments?user.id=879f296733114f6f844a98aaefe35e92&scope.project.id=36a3ae7bab25471fb989136150b6438c&role.id=d00b78c32dbf4cc7925a55bb621d8fcf => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:13:51.346452 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ef0ce-acab-4dee-b3ce-43d05145c0ef None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:51.347010 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ef0ce-acab-4dee-b3ce-43d05145c0ef None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:51.347010 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ef0ce-acab-4dee-b3ce-43d05145c0ef None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:51.348601 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 430/871] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:51.354681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec1fa74-1691-4afa-9a33-5fa97ae01967 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:51.355101 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec1fa74-1691-4afa-9a33-5fa97ae01967 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:51.355337 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec1fa74-1691-4afa-9a33-5fa97ae01967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:51.373795 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-0ec1fa74-1691-4afa-9a33-5fa97ae01967 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:51.375093 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-0ec1fa74-1691-4afa-9a33-5fa97ae01967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:51.442947 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 442/872] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:51 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:13:51.713159 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ef0ce-acab-4dee-b3ce-43d05145c0ef None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:51.745738 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:51.829294 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ef0ce-acab-4dee-b3ce-43d05145c0ef None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:51.865413 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b059a6-9859-49a1-b98d-5906fe259cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:51.866047 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b059a6-9859-49a1-b98d-5906fe259cc8 None admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:51.866328 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b059a6-9859-49a1-b98d-5906fe259cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:51.866601 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b059a6-9859-49a1-b98d-5906fe259cc8 None admin] PATH_INFO: `/v3/services` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:51.867325 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b059a6-9859-49a1-b98d-5906fe259cc8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:51.869292 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b059a6-9859-49a1-b98d-5906fe259cc8 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:51.892141 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 431/873] 10.222.0.52 () {60 vars in 1141 bytes} [Sat Sep 24 14:13:51 2022] POST /identity/v3/services => generated 251 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:13:51.990217 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabb923-585b-4c9d-a5c5-74c89078de18 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:51.990533 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabb923-585b-4c9d-a5c5-74c89078de18 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:51.990696 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabb923-585b-4c9d-a5c5-74c89078de18 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:51.992331 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 443/874] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:51.998624 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51efd9e-e525-49dc-a407-38cb2b6405c7 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:51.998960 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51efd9e-e525-49dc-a407-38cb2b6405c7 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:51.999242 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51efd9e-e525-49dc-a407-38cb2b6405c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:52.018743 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-e51efd9e-e525-49dc-a407-38cb2b6405c7 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:52.021674 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-e51efd9e-e525-49dc-a407-38cb2b6405c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:52.108430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 432/875] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:51 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:13:52.375747 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edabb923-585b-4c9d-a5c5-74c89078de18 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:52.477413 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:52.481925 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edabb923-585b-4c9d-a5c5-74c89078de18 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:52.518229 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec943be-2497-4a01-8dca-648b47c91ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:52.518891 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec943be-2497-4a01-8dca-648b47c91ff7 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:52.519170 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec943be-2497-4a01-8dca-648b47c91ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:52.519434 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec943be-2497-4a01-8dca-648b47c91ff7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:52.531093 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec943be-2497-4a01-8dca-648b47c91ff7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:13:52.536357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-0ec943be-2497-4a01-8dca-648b47c91ff7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 14:13:52.536357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:52.536357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:52.536357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:52.536357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:52.536357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:52.536357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:52.536357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:52.536357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:52.536357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:52.536357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:52.536357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:52.536357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:13:52.536357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:13:52.536357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:13:52.536357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:13:52.537423 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:13:52.537423 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:13:52.537423 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:13:52.537423 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:13:52.537423 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:52.537423 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:52.537423 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:52.537423 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:52.537423 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:52.537423 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:52.537423 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:52.537423 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:52.537423 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:52.537423 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:52.537423 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:52.537423 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:52.538314 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:52.538314 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:52.538314 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:52.538314 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:52.538314 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:13:52.538314 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:13:52.538314 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:13:52.538314 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:13:52.538314 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:13:52.538314 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:13:52.538314 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 14:13:52.538314 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:52.538314 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 444/876] 10.222.0.52 () {56 vars in 1120 bytes} [Sat Sep 24 14:13:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:52.544753 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51efd9e-e525-49dc-a407-38cb2b6405c7 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:52.566465 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:52.655976 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51efd9e-e525-49dc-a407-38cb2b6405c7 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:52.664248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad3a214-425d-4317-beb5-76d80c161a0f None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:52.664530 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad3a214-425d-4317-beb5-76d80c161a0f None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:52.664796 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad3a214-425d-4317-beb5-76d80c161a0f None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:52.668481 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 445/877] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:52.673586 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169142c0-c9f6-438d-be5d-2dbc24c8701c None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:52.673864 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169142c0-c9f6-438d-be5d-2dbc24c8701c None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:52.674138 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169142c0-c9f6-438d-be5d-2dbc24c8701c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:52.692150 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-169142c0-c9f6-438d-be5d-2dbc24c8701c None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:52.693329 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-169142c0-c9f6-438d-be5d-2dbc24c8701c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:52.693916 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbe1418-262f-491c-ba49-061193067d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:52.694532 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe1418-262f-491c-ba49-061193067d49 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:52.694844 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe1418-262f-491c-ba49-061193067d49 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:52.695100 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe1418-262f-491c-ba49-061193067d49 None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:52.695955 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbe1418-262f-491c-ba49-061193067d49 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:52.697589 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbe1418-262f-491c-ba49-061193067d49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:52.698969 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbe1418-262f-491c-ba49-061193067d49 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:52.713205 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 433/878] 10.222.0.52 () {56 vars in 1115 bytes} [Sat Sep 24 14:13:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:52.713205 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224abf3-dbe8-4a71-818e-381c317fd8b6 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:52.713205 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224abf3-dbe8-4a71-818e-381c317fd8b6 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:52.713205 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224abf3-dbe8-4a71-818e-381c317fd8b6 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:52.714090 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 434/879] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:52.719739 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0224abf3-dbe8-4a71-818e-381c317fd8b6 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:52.766772 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 446/880] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:52 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:13:52.768041 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c60a0-e76b-4c24-8d9a-e0bb979d7ff7 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:52.768322 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c60a0-e76b-4c24-8d9a-e0bb979d7ff7 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:52.768591 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c60a0-e76b-4c24-8d9a-e0bb979d7ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:52.790704 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-473c60a0-e76b-4c24-8d9a-e0bb979d7ff7 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:52.792254 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-473c60a0-e76b-4c24-8d9a-e0bb979d7ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:52.838949 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0224abf3-dbe8-4a71-818e-381c317fd8b6 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:52.851182 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:52.878914 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 447/881] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:52 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:13:52.881266 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4271deb9-6a08-4524-a0a1-3989eb41adae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:52.881266 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271deb9-6a08-4524-a0a1-3989eb41adae None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:52.881266 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271deb9-6a08-4524-a0a1-3989eb41adae None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:52.881266 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271deb9-6a08-4524-a0a1-3989eb41adae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:52.892732 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4271deb9-6a08-4524-a0a1-3989eb41adae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:52.894169 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4271deb9-6a08-4524-a0a1-3989eb41adae None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:52.910417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-4271deb9-6a08-4524-a0a1-3989eb41adae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:52.910417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:52.910417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:52.910417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:52.910417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:52.910417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:52.910417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:52.910417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:52.910417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:52.910417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:52.910417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:52.910417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:52.910417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:52.910417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:52.910417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:52.910417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:52.911549 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:52.911549 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:52.911549 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:52.911549 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:52.911549 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:52.911549 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:52.911549 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:52.911549 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:52.911549 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:52.911549 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:52.911549 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:52.911549 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:52.911549 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:52.911549 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:52.911549 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:52.911549 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:52.911549 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:52.913168 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:52.913168 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:52.913168 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:52.913168 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 435/882] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:52.917508 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473c60a0-e76b-4c24-8d9a-e0bb979d7ff7 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:52.939568 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:53.025825 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473c60a0-e76b-4c24-8d9a-e0bb979d7ff7 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:53.041222 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4271deb9-6a08-4524-a0a1-3989eb41adae None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:53.065893 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e4376e-078f-4e05-8167-74eb5c5a9cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:53.066507 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e4376e-078f-4e05-8167-74eb5c5a9cdd None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:53.066785 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e4376e-078f-4e05-8167-74eb5c5a9cdd None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:53.067100 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e4376e-078f-4e05-8167-74eb5c5a9cdd None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:53.067997 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e4376e-078f-4e05-8167-74eb5c5a9cdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:53.068296 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e4376e-078f-4e05-8167-74eb5c5a9cdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:53.070367 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e4376e-078f-4e05-8167-74eb5c5a9cdd None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:53.091118 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 448/883] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:53.092005 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e4376e-078f-4e05-8167-74eb5c5a9cdd None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:53.157464 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4271deb9-6a08-4524-a0a1-3989eb41adae None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:53.190705 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:53.195695 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:53.197601 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353b261-ec47-4bf6-87c9-bd7ebff1ae2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:53.198238 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353b261-ec47-4bf6-87c9-bd7ebff1ae2e None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:53.198518 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353b261-ec47-4bf6-87c9-bd7ebff1ae2e None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:53.198852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353b261-ec47-4bf6-87c9-bd7ebff1ae2e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:53.199395 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b353b261-ec47-4bf6-87c9-bd7ebff1ae2e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:53.200756 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b353b261-ec47-4bf6-87c9-bd7ebff1ae2e None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:53.209569 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-b353b261-ec47-4bf6-87c9-bd7ebff1ae2e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 14:13:53.209569 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:53.209569 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:13:53.209569 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:13:53.209569 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:13:53.209569 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:13:53.209569 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:13:53.209569 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:13:53.209569 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 14:13:53.209569 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:53.209569 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:13:53.209569 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:53.209569 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:53.209569 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:53.209569 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:53.209569 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:53.209569 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:53.210587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:53.210587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:53.210587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:53.210587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:53.210587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:53.210587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:53.210587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:13:53.210587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:13:53.210587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:13:53.210587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:13:53.210587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:53.210587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:53.210587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:53.210587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:53.210587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:53.210587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:53.211460 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:53.211460 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:53.211460 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:53.211460 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:53.211460 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:53.211460 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:53.211460 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:53.211460 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:53.211460 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:53.211460 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:53.211460 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:13:53.211460 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:13:53.211460 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 14:13:53.211460 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:53.211460 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 436/884] 10.222.0.52 () {56 vars in 1126 bytes} [Sat Sep 24 14:13:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:13:53.213289 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353b261-ec47-4bf6-87c9-bd7ebff1ae2e None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:53.221712 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e4376e-078f-4e05-8167-74eb5c5a9cdd None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:53.262281 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca33aea-a618-4235-ba6e-26aea4aece12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:53.263226 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca33aea-a618-4235-ba6e-26aea4aece12 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:53.263509 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca33aea-a618-4235-ba6e-26aea4aece12 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:53.263783 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca33aea-a618-4235-ba6e-26aea4aece12 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:53.272126 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca33aea-a618-4235-ba6e-26aea4aece12 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:53.273569 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca33aea-a618-4235-ba6e-26aea4aece12 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:53.285726 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-eca33aea-a618-4235-ba6e-26aea4aece12 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 14:13:53.285726 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:53.285726 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:53.285726 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:53.285726 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:53.285726 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:53.285726 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:53.285726 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:53.285726 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:53.285726 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:53.285726 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:53.285726 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:53.285726 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:13:53.285726 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:13:53.285726 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:13:53.285726 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:13:53.286736 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:53.286736 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:53.286736 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:13:53.286736 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:53.286736 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:13:53.286736 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:53.286736 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:53.286736 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:53.286736 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:53.286736 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:53.286736 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:53.286736 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:53.286736 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:53.286736 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:53.286736 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:53.286736 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:53.286736 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:53.286736 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:53.287713 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:53.287713 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:53.287713 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:13:53.287713 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:13:53.287713 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:13:53.287713 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:13:53.287713 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:13:53.287713 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:13:53.287713 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 14:13:53.287713 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:53.287713 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 449/885] 10.222.0.52 () {56 vars in 1117 bytes} [Sat Sep 24 14:13:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:53.287713 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca33aea-a618-4235-ba6e-26aea4aece12 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:53.325762 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353b261-ec47-4bf6-87c9-bd7ebff1ae2e None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:53.363529 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6175e0-665b-41f4-9c46-7324df1d64d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:53.364131 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6175e0-665b-41f4-9c46-7324df1d64d6 None admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:53.364413 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6175e0-665b-41f4-9c46-7324df1d64d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:53.364682 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6175e0-665b-41f4-9c46-7324df1d64d6 None admin] PATH_INFO: `/v3/services` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:53.365740 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6175e0-665b-41f4-9c46-7324df1d64d6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:53.367478 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6175e0-665b-41f4-9c46-7324df1d64d6 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:53.387504 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 437/886] 10.222.0.52 () {60 vars in 1141 bytes} [Sat Sep 24 14:13:53 2022] POST /identity/v3/services => generated 269 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:13:53.388657 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6175e0-665b-41f4-9c46-7324df1d64d6 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:53.392324 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca33aea-a618-4235-ba6e-26aea4aece12 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:53.449513 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a6f7c3-20b8-40e8-9f16-014fec3c0822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:53.450068 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6f7c3-20b8-40e8-9f16-014fec3c0822 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:53.450444 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6f7c3-20b8-40e8-9f16-014fec3c0822 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:53.450827 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6f7c3-20b8-40e8-9f16-014fec3c0822 None admin] PATH_INFO: `/v3/services` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:53.452020 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a6f7c3-20b8-40e8-9f16-014fec3c0822 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:53.452020 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a6f7c3-20b8-40e8-9f16-014fec3c0822 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:53.453672 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a6f7c3-20b8-40e8-9f16-014fec3c0822 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:53.455972 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:53.467213 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 450/887] 10.222.0.52 () {56 vars in 1122 bytes} [Sat Sep 24 14:13:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:53.468326 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a6f7c3-20b8-40e8-9f16-014fec3c0822 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:53.469736 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:53.523726 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6175e0-665b-41f4-9c46-7324df1d64d6 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:53.566452 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db67a2d-eaa6-4744-b508-8348ca8f13c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:53.567132 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db67a2d-eaa6-4744-b508-8348ca8f13c3 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:53.567132 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db67a2d-eaa6-4744-b508-8348ca8f13c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:53.567132 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db67a2d-eaa6-4744-b508-8348ca8f13c3 None admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:53.567842 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db67a2d-eaa6-4744-b508-8348ca8f13c3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:53.569214 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db67a2d-eaa6-4744-b508-8348ca8f13c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:53.569587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db67a2d-eaa6-4744-b508-8348ca8f13c3 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:53.591002 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 438/888] 10.222.0.52 () {56 vars in 1149 bytes} [Sat Sep 24 14:13:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:53.596365 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db67a2d-eaa6-4744-b508-8348ca8f13c3 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:53.598329 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a6f7c3-20b8-40e8-9f16-014fec3c0822 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:53.639099 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada37f4-2225-4277-84a8-e5e4b0bb386a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:53.639709 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada37f4-2225-4277-84a8-e5e4b0bb386a None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:53.639984 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada37f4-2225-4277-84a8-e5e4b0bb386a None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:53.640249 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada37f4-2225-4277-84a8-e5e4b0bb386a None admin] PATH_INFO: `/v3/services` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:53.640874 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada37f4-2225-4277-84a8-e5e4b0bb386a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:53.641174 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada37f4-2225-4277-84a8-e5e4b0bb386a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:53.642535 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada37f4-2225-4277-84a8-e5e4b0bb386a None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:53.651541 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 451/889] 10.222.0.52 () {56 vars in 1122 bytes} [Sat Sep 24 14:13:53 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:53.718767 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db67a2d-eaa6-4744-b508-8348ca8f13c3 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:53.735729 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:53.758548 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34716c7-59c9-44e6-b119-6e435e6ee5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:53.760148 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34716c7-59c9-44e6-b119-6e435e6ee5a4 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:53.760639 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34716c7-59c9-44e6-b119-6e435e6ee5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:53.761116 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34716c7-59c9-44e6-b119-6e435e6ee5a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:53.837445 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34716c7-59c9-44e6-b119-6e435e6ee5a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:53.838919 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34716c7-59c9-44e6-b119-6e435e6ee5a4 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:53.940205 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 439/890] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:13:53 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 24 14:13:53.945839 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada37f4-2225-4277-84a8-e5e4b0bb386a None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:54.016486 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:54.058571 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada37f4-2225-4277-84a8-e5e4b0bb386a None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:54.094390 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9566d6f-5e8e-416c-920c-3917f893b1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:54.095054 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9566d6f-5e8e-416c-920c-3917f893b1df None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:54.095335 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9566d6f-5e8e-416c-920c-3917f893b1df None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:54.095606 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9566d6f-5e8e-416c-920c-3917f893b1df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:54.108278 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9566d6f-5e8e-416c-920c-3917f893b1df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:54.109790 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9566d6f-5e8e-416c-920c-3917f893b1df None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:54.127097 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-f9566d6f-5e8e-416c-920c-3917f893b1df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:54.127097 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:54.127097 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:54.127097 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:54.127097 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:54.127097 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:54.127097 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:54.127097 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:54.127097 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:54.127097 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:54.127097 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:54.127097 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:54.127097 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:54.127097 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:54.127097 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:54.127097 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:54.128098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:54.128098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:54.128098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:54.128098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:54.128098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:54.128098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:54.128098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:54.128098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:54.128098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:54.128098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:54.128098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:54.128098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:54.128098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:54.128098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:54.128098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:54.128098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:54.128098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:54.128970 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:54.128970 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:54.128970 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:54.128970 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 452/891] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:54.136173 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34716c7-59c9-44e6-b119-6e435e6ee5a4 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:54.152228 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:54.228538 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc815d29-d62d-449e-a56c-10ae836ec454 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:54.228895 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc815d29-d62d-449e-a56c-10ae836ec454 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:54.229104 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc815d29-d62d-449e-a56c-10ae836ec454 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:54.231870 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 453/892] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:13:54.235979 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab508fc-6700-4772-b80f-2cc758aee383 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:54.236265 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab508fc-6700-4772-b80f-2cc758aee383 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:54.236536 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab508fc-6700-4772-b80f-2cc758aee383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:54.245036 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34716c7-59c9-44e6-b119-6e435e6ee5a4 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:54.255593 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-bab508fc-6700-4772-b80f-2cc758aee383 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:54.256756 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-bab508fc-6700-4772-b80f-2cc758aee383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:54.291205 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cc1c59-7135-4219-ab9c-0f9a58d73099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:54.291742 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cc1c59-7135-4219-ab9c-0f9a58d73099 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:54.292015 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cc1c59-7135-4219-ab9c-0f9a58d73099 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:54.292282 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cc1c59-7135-4219-ab9c-0f9a58d73099 None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:54.292924 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cc1c59-7135-4219-ab9c-0f9a58d73099 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:54.293221 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cc1c59-7135-4219-ab9c-0f9a58d73099 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:54.294863 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cc1c59-7135-4219-ab9c-0f9a58d73099 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:54.307027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 440/893] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:54.315773 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cc1c59-7135-4219-ab9c-0f9a58d73099 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:54.337730 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 454/894] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:54 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:13:54.397959 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:54.426908 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cc1c59-7135-4219-ab9c-0f9a58d73099 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:54.463000 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da82c68-ced6-4a7f-adb0-d2fe414f27bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:54.463629 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da82c68-ced6-4a7f-adb0-d2fe414f27bf None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:54.463902 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da82c68-ced6-4a7f-adb0-d2fe414f27bf None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:54.464166 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da82c68-ced6-4a7f-adb0-d2fe414f27bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:54.471001 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da82c68-ced6-4a7f-adb0-d2fe414f27bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:54.472435 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da82c68-ced6-4a7f-adb0-d2fe414f27bf None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:54.484203 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-1da82c68-ced6-4a7f-adb0-d2fe414f27bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:54.484203 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:54.484203 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:54.484203 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:54.484203 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:54.484203 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:54.484203 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:54.484203 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:54.484203 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:54.484203 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:54.484203 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:54.484203 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:54.484203 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:13:54.484203 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:13:54.484203 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:13:54.485094 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:13:54.485094 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:54.485094 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:54.485094 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:54.485094 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:54.485094 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:54.485094 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:54.485094 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:54.485094 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:54.485094 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:54.485094 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:54.485094 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:54.485094 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:54.485094 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:54.485094 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:54.485094 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:54.485094 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:54.485962 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:13:54.485962 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:13:54.485962 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:13:54.485962 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:13:54.485962 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:13:54.485962 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:13:54.485962 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:54.485962 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:54.485962 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 441/895] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:13:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:54.491268 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab508fc-6700-4772-b80f-2cc758aee383 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:54.543809 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:54.599755 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab508fc-6700-4772-b80f-2cc758aee383 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:54.612748 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da82c68-ced6-4a7f-adb0-d2fe414f27bf None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:54.643055 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b2592-10e0-4026-9460-68077fd14ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:54.644025 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b2592-10e0-4026-9460-68077fd14ccd None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:54.644524 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b2592-10e0-4026-9460-68077fd14ccd None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:54.644992 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b2592-10e0-4026-9460-68077fd14ccd None admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:54.645897 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629b2592-10e0-4026-9460-68077fd14ccd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:54.646391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629b2592-10e0-4026-9460-68077fd14ccd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:54.648045 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629b2592-10e0-4026-9460-68077fd14ccd None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:54.661026 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 455/896] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 14:13:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:54.668065 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b2592-10e0-4026-9460-68077fd14ccd None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:54.735657 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da82c68-ced6-4a7f-adb0-d2fe414f27bf None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:54.773667 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4f51f-71e4-40f4-adbe-49b011ea7630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:54.774293 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4f51f-71e4-40f4-adbe-49b011ea7630 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:54.775178 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4f51f-71e4-40f4-adbe-49b011ea7630 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:54.775409 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4f51f-71e4-40f4-adbe-49b011ea7630 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:54.775987 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f4f51f-71e4-40f4-adbe-49b011ea7630 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:54.777826 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f4f51f-71e4-40f4-adbe-49b011ea7630 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:54.780067 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:54.789215 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-26f4f51f-71e4-40f4-adbe-49b011ea7630 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 14:13:54.789215 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:54.789215 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:13:54.789215 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:13:54.789215 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:13:54.789215 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:13:54.789215 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:13:54.789215 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:13:54.789215 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 14:13:54.789215 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:54.789215 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:13:54.789215 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:54.789215 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:54.789215 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:54.789215 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:54.789215 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:54.789215 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:54.790773 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:54.790773 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:54.790773 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:54.790773 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:54.790773 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:54.790773 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:54.790773 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:13:54.790773 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:13:54.790773 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:13:54.790773 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:13:54.790773 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:54.790773 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:54.790773 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:54.790773 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:54.790773 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:54.790773 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:54.791657 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:54.791657 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:54.791657 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:54.791657 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:54.791657 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:54.791657 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:54.791657 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:54.791657 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:54.791657 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:54.791657 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:54.791657 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:13:54.791657 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:13:54.791657 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 14:13:54.791657 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:54.796898 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b2592-10e0-4026-9460-68077fd14ccd None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:54.798925 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 442/897] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 14:13:54 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:13:54.800691 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4f51f-71e4-40f4-adbe-49b011ea7630 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:54.806621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:54.843945 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94638218-b5e4-40dd-8d2e-32708fd0d449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:54.845140 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94638218-b5e4-40dd-8d2e-32708fd0d449 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:54.845653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94638218-b5e4-40dd-8d2e-32708fd0d449 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:54.846144 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94638218-b5e4-40dd-8d2e-32708fd0d449 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:54.847305 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94638218-b5e4-40dd-8d2e-32708fd0d449 None admin] RBAC: Adding query filter params (role.id=30791f3f2a7841d2a80aab92cf4f23f5, scope.project.id=36a3ae7bab25471fb989136150b6438c, user.id=82328b7fd50649a485c6eaf27c5597c9) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:54.848015 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94638218-b5e4-40dd-8d2e-32708fd0d449 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:54.849704 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94638218-b5e4-40dd-8d2e-32708fd0d449 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:54.863279 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 456/898] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:13:54 2022] GET /identity/v3/role_assignments?user.id=82328b7fd50649a485c6eaf27c5597c9&scope.project.id=36a3ae7bab25471fb989136150b6438c&role.id=30791f3f2a7841d2a80aab92cf4f23f5 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:13:54.924427 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4f51f-71e4-40f4-adbe-49b011ea7630 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:54.963315 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73901fcc-9bb3-4f65-a232-ec2683a2502e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:54.963940 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73901fcc-9bb3-4f65-a232-ec2683a2502e None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:54.964216 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73901fcc-9bb3-4f65-a232-ec2683a2502e None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:54.964482 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73901fcc-9bb3-4f65-a232-ec2683a2502e None admin] PATH_INFO: `/v3/services` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:54.965116 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73901fcc-9bb3-4f65-a232-ec2683a2502e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:54.965414 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73901fcc-9bb3-4f65-a232-ec2683a2502e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:54.966772 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73901fcc-9bb3-4f65-a232-ec2683a2502e None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:54.976792 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 443/899] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 14:13:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:54.979979 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94638218-b5e4-40dd-8d2e-32708fd0d449 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:55.094803 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94638218-b5e4-40dd-8d2e-32708fd0d449 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:55.129948 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f9566-b0c3-4bee-9508-1510fb29a143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:55.130567 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f9566-b0c3-4bee-9508-1510fb29a143 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:55.130866 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f9566-b0c3-4bee-9508-1510fb29a143 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:55.131144 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f9566-b0c3-4bee-9508-1510fb29a143 None admin] PATH_INFO: `/v3/services` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:55.131773 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f9566-b0c3-4bee-9508-1510fb29a143 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:55.132075 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f9566-b0c3-4bee-9508-1510fb29a143 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:55.133438 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f9566-b0c3-4bee-9508-1510fb29a143 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:55.139207 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 457/900] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 14:13:54 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:55.144792 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73901fcc-9bb3-4f65-a232-ec2683a2502e None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:55.231331 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:55.253837 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73901fcc-9bb3-4f65-a232-ec2683a2502e None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:55.289690 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9689d3c2-ca2b-4a80-b5b7-8008a4a2e830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:55.290846 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689d3c2-ca2b-4a80-b5b7-8008a4a2e830 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:55.291132 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689d3c2-ca2b-4a80-b5b7-8008a4a2e830 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:55.291399 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689d3c2-ca2b-4a80-b5b7-8008a4a2e830 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:55.292092 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9689d3c2-ca2b-4a80-b5b7-8008a4a2e830 None admin] RBAC: Adding query filter params (interface=public, service_id=7ce69c6ea078408fbb78fdb9ee1e594b, region_id=RegionOne) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:55.292391 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9689d3c2-ca2b-4a80-b5b7-8008a4a2e830 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:55.293731 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9689d3c2-ca2b-4a80-b5b7-8008a4a2e830 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:55.302929 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 444/901] 10.222.0.52 () {56 vars in 1251 bytes} [Sat Sep 24 14:13:55 2022] GET /identity/v3/endpoints?service_id=7ce69c6ea078408fbb78fdb9ee1e594b&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:55.305025 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f9566-b0c3-4bee-9508-1510fb29a143 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:55.340513 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:55.412638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f9566-b0c3-4bee-9508-1510fb29a143 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:55.448608 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926d497f-ab91-4600-9f8f-5948b7c9ecd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:55.449562 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d497f-ab91-4600-9f8f-5948b7c9ecd4 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:55.450116 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d497f-ab91-4600-9f8f-5948b7c9ecd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:55.450598 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d497f-ab91-4600-9f8f-5948b7c9ecd4 None admin] PATH_INFO: `/v3/services` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:55.451442 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926d497f-ab91-4600-9f8f-5948b7c9ecd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:55.453000 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926d497f-ab91-4600-9f8f-5948b7c9ecd4 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:55.459311 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 458/902] 10.222.0.52 () {56 vars in 1087 bytes} [Sat Sep 24 14:13:55 2022] GET /identity/v3/services => generated 1268 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:13:55.770795 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d420d5b-95af-47b8-ad1a-74070b825144 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:55.771321 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d420d5b-95af-47b8-ad1a-74070b825144 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:55.771321 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d420d5b-95af-47b8-ad1a-74070b825144 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:55.772901 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 445/903] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:55.778999 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e1b89-6ae3-4180-adcc-e04e34316922 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:55.779519 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e1b89-6ae3-4180-adcc-e04e34316922 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:55.779519 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e1b89-6ae3-4180-adcc-e04e34316922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:55.800186 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-0f7e1b89-6ae3-4180-adcc-e04e34316922 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:55.801391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-0f7e1b89-6ae3-4180-adcc-e04e34316922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:55.885448 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 459/904] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:55 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:13:55.943731 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d066801-e828-449f-8fd1-20694fa61f1f None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:55.944054 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d066801-e828-449f-8fd1-20694fa61f1f None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:55.944279 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d066801-e828-449f-8fd1-20694fa61f1f None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:55.945875 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 446/905] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:55.951629 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affeb92b-2ef7-4728-9711-8c8f165dc622 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:55.951918 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affeb92b-2ef7-4728-9711-8c8f165dc622 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:55.952196 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affeb92b-2ef7-4728-9711-8c8f165dc622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:55.971546 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-affeb92b-2ef7-4728-9711-8c8f165dc622 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:55.972723 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-affeb92b-2ef7-4728-9711-8c8f165dc622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:55.996649 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:56.043715 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 460/906] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:55 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:13:56.159062 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d066801-e828-449f-8fd1-20694fa61f1f None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:56.267804 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d066801-e828-449f-8fd1-20694fa61f1f None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:56.293787 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:56.302718 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa9502-111a-4d84-a816-7148125cfea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:56.303408 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa9502-111a-4d84-a816-7148125cfea8 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:56.303734 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa9502-111a-4d84-a816-7148125cfea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:56.304116 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa9502-111a-4d84-a816-7148125cfea8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:56.304709 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa9502-111a-4d84-a816-7148125cfea8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:56.307136 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa9502-111a-4d84-a816-7148125cfea8 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:56.316440 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-2aaa9502-111a-4d84-a816-7148125cfea8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 14:13:56.316440 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:56.316440 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:13:56.316440 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:13:56.316440 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:13:56.316440 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:13:56.316440 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:13:56.316440 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:13:56.316440 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 14:13:56.316440 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:56.316440 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:13:56.316440 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:56.316440 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:56.316440 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:56.316440 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:56.316440 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:56.317355 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:56.317355 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:56.317355 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:56.317355 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:56.317355 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:56.317355 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:56.317355 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:56.317355 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:13:56.317355 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:13:56.317355 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:13:56.317355 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:13:56.317355 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:56.317355 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:56.317355 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:56.317355 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:56.317355 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:56.317355 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:56.318221 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:56.318221 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:56.318221 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:56.318221 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:56.318221 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:56.318221 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:56.318221 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:56.318221 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:56.318221 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:56.318221 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:56.318221 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:13:56.318221 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:13:56.318221 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 14:13:56.318221 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:56.318221 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 447/907] 10.222.0.52 () {56 vars in 1132 bytes} [Sat Sep 24 14:13:56 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:56.321801 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affeb92b-2ef7-4728-9711-8c8f165dc622 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:56.324318 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa9502-111a-4d84-a816-7148125cfea8 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:56.439937 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa9502-111a-4d84-a816-7148125cfea8 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:56.443897 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affeb92b-2ef7-4728-9711-8c8f165dc622 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:56.474500 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d8d37d-1242-45d0-aa60-0643148cd5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:56.475241 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8d37d-1242-45d0-aa60-0643148cd5f0 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:56.475524 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8d37d-1242-45d0-aa60-0643148cd5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:56.475823 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8d37d-1242-45d0-aa60-0643148cd5f0 None admin] PATH_INFO: `/v3/services` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:56.476516 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d8d37d-1242-45d0-aa60-0643148cd5f0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:56.476819 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d8d37d-1242-45d0-aa60-0643148cd5f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:56.478797 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d8d37d-1242-45d0-aa60-0643148cd5f0 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:56.483772 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 448/908] 10.222.0.52 () {56 vars in 1126 bytes} [Sat Sep 24 14:13:56 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:13:56.489028 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d8d37d-1242-45d0-aa60-0643148cd5f0 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:56.489331 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb961ebb-9991-4b6c-8253-dcb80473f36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:56.490105 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb961ebb-9991-4b6c-8253-dcb80473f36e None admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:56.490574 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb961ebb-9991-4b6c-8253-dcb80473f36e None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:56.491065 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb961ebb-9991-4b6c-8253-dcb80473f36e None admin] PATH_INFO: `/v3/services` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:56.492389 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb961ebb-9991-4b6c-8253-dcb80473f36e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:56.495437 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb961ebb-9991-4b6c-8253-dcb80473f36e None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:56.514093 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 461/909] 10.222.0.52 () {60 vars in 1141 bytes} [Sat Sep 24 14:13:56 2022] POST /identity/v3/services => generated 241 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:13:56.556829 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:56.604864 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d8d37d-1242-45d0-aa60-0643148cd5f0 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:56.648234 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a32c79c-3512-4841-b2e2-229f910b91e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:56.648857 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32c79c-3512-4841-b2e2-229f910b91e5 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:56.649138 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32c79c-3512-4841-b2e2-229f910b91e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:56.649460 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32c79c-3512-4841-b2e2-229f910b91e5 None admin] PATH_INFO: `/v3/services` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:56.650105 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a32c79c-3512-4841-b2e2-229f910b91e5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:56.650408 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a32c79c-3512-4841-b2e2-229f910b91e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:56.651852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a32c79c-3512-4841-b2e2-229f910b91e5 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:56.657887 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 449/910] 10.222.0.52 () {56 vars in 1126 bytes} [Sat Sep 24 14:13:56 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:56.664079 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb961ebb-9991-4b6c-8253-dcb80473f36e None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:56.687186 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:56.776696 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb961ebb-9991-4b6c-8253-dcb80473f36e None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:56.812945 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda6739c-9c29-47a8-8f0c-a306e6d234b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:56.813661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6739c-9c29-47a8-8f0c-a306e6d234b4 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:56.813821 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6739c-9c29-47a8-8f0c-a306e6d234b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:56.813990 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6739c-9c29-47a8-8f0c-a306e6d234b4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:56.814693 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda6739c-9c29-47a8-8f0c-a306e6d234b4 None admin] RBAC: Adding query filter params (interface=public, service_id=e348a991cdac483480beef296ab05d47, region_id=RegionOne) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:56.815045 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda6739c-9c29-47a8-8f0c-a306e6d234b4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:56.818347 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda6739c-9c29-47a8-8f0c-a306e6d234b4 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:56.824283 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 462/911] 10.222.0.52 () {56 vars in 1251 bytes} [Sat Sep 24 14:13:56 2022] GET /identity/v3/endpoints?service_id=e348a991cdac483480beef296ab05d47&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:56.831987 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a32c79c-3512-4841-b2e2-229f910b91e5 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:56.934620 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a32c79c-3512-4841-b2e2-229f910b91e5 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:56.949184 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:56.969671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b00f268-7071-42c4-a7a8-ec48710c9746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:56.970284 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b00f268-7071-42c4-a7a8-ec48710c9746 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:56.970565 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b00f268-7071-42c4-a7a8-ec48710c9746 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:56.970865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b00f268-7071-42c4-a7a8-ec48710c9746 None admin] PATH_INFO: `/v3/services` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:56.971482 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b00f268-7071-42c4-a7a8-ec48710c9746 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:56.972849 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b00f268-7071-42c4-a7a8-ec48710c9746 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:56.979392 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 450/912] 10.222.0.52 () {56 vars in 1087 bytes} [Sat Sep 24 14:13:56 2022] GET /identity/v3/services => generated 1497 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:13:57.178277 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c92e8-1407-4ca1-8259-ad463a2b3369 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:57.179026 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c92e8-1407-4ca1-8259-ad463a2b3369 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:57.179475 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c92e8-1407-4ca1-8259-ad463a2b3369 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:57.181006 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 463/913] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:57.186148 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0a617-3181-400a-95cc-95bcd5eabc51 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:57.186442 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0a617-3181-400a-95cc-95bcd5eabc51 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:57.186737 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0a617-3181-400a-95cc-95bcd5eabc51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:57.207168 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-7df0a617-3181-400a-95cc-95bcd5eabc51 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:57.208707 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-7df0a617-3181-400a-95cc-95bcd5eabc51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:57.279401 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 451/914] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:13:57.549413 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c92e8-1407-4ca1-8259-ad463a2b3369 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:57.642826 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:57.663771 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c92e8-1407-4ca1-8259-ad463a2b3369 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:57.699159 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c337fe-08a5-412c-bd83-d7bd08f8659c None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:57.699481 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c337fe-08a5-412c-bd83-d7bd08f8659c None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:57.699644 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c337fe-08a5-412c-bd83-d7bd08f8659c None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:57.699803 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf471357-d060-4296-b489-d077535f52b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:57.700442 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf471357-d060-4296-b489-d077535f52b8 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:57.700717 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf471357-d060-4296-b489-d077535f52b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:57.701031 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf471357-d060-4296-b489-d077535f52b8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:57.702310 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 452/915] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:57.706285 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00717b5-76e8-43ed-97c6-013ec4a30684 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:57.706563 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00717b5-76e8-43ed-97c6-013ec4a30684 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:57.706849 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00717b5-76e8-43ed-97c6-013ec4a30684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:57.713206 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf471357-d060-4296-b489-d077535f52b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:13:57.718290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-cf471357-d060-4296-b489-d077535f52b8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 14:13:57.718290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:57.718290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:57.718290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:57.718290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:57.718290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:57.718290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:57.718290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:57.718290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:57.718290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:57.718290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:57.718290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:57.718290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:13:57.718290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:13:57.718290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:13:57.718290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:13:57.719265 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:13:57.719265 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:13:57.719265 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:13:57.719265 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:13:57.719265 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:57.719265 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:57.719265 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:57.719265 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:57.719265 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:57.719265 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:57.719265 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:57.719265 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:57.719265 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:57.719265 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:57.719265 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:57.719265 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:57.720099 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:57.720099 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:57.720099 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:57.720099 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:57.720099 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:13:57.720099 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:13:57.720099 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:13:57.720099 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:13:57.720099 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:13:57.720099 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:13:57.720099 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 14:13:57.720099 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:57.720099 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 464/916] 10.222.0.52 () {56 vars in 1120 bytes} [Sat Sep 24 14:13:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:57.725457 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf471357-d060-4296-b489-d077535f52b8 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:57.726954 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-b00717b5-76e8-43ed-97c6-013ec4a30684 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:57.728277 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-b00717b5-76e8-43ed-97c6-013ec4a30684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:57.823235 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 453/917] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:13:57.859340 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf471357-d060-4296-b489-d077535f52b8 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:57.895433 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97266ace-e35a-49e8-95e1-9695cef85063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:57.896057 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97266ace-e35a-49e8-95e1-9695cef85063 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:57.896339 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97266ace-e35a-49e8-95e1-9695cef85063 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:57.896609 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97266ace-e35a-49e8-95e1-9695cef85063 None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:57.897295 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97266ace-e35a-49e8-95e1-9695cef85063 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:57.897594 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97266ace-e35a-49e8-95e1-9695cef85063 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:57.899027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97266ace-e35a-49e8-95e1-9695cef85063 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:57.909827 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 465/918] 10.222.0.52 () {56 vars in 1115 bytes} [Sat Sep 24 14:13:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:13:57.917281 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00717b5-76e8-43ed-97c6-013ec4a30684 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:57.960452 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:58.025973 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00717b5-76e8-43ed-97c6-013ec4a30684 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:58.060240 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73732ed-2285-42e7-9edf-a3deae4c5e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:58.060860 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73732ed-2285-42e7-9edf-a3deae4c5e10 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:58.061136 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73732ed-2285-42e7-9edf-a3deae4c5e10 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:58.061401 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73732ed-2285-42e7-9edf-a3deae4c5e10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:58.071311 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73732ed-2285-42e7-9edf-a3deae4c5e10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:58.072731 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73732ed-2285-42e7-9edf-a3deae4c5e10 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:58.086970 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-b73732ed-2285-42e7-9edf-a3deae4c5e10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:58.086970 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:58.086970 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:58.086970 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:58.086970 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:58.086970 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:58.086970 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:58.086970 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:58.086970 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:58.086970 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:58.086970 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:58.086970 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:58.086970 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:58.086970 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:58.086970 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:58.086970 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:58.087882 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:58.087882 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:58.087882 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:58.087882 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:58.087882 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:58.087882 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:58.087882 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:58.087882 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:58.087882 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:58.087882 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:58.087882 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:58.087882 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:58.087882 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:58.087882 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:58.087882 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:58.087882 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:58.087882 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:58.088781 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:58.088781 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:58.088781 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:58.088781 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 454/919] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:58.095892 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97266ace-e35a-49e8-95e1-9695cef85063 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:58.100694 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:58.154094 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73732ed-2285-42e7-9edf-a3deae4c5e10 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:58.205406 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97266ace-e35a-49e8-95e1-9695cef85063 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:58.246778 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8819f6-2214-40da-9ead-8582257d07f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:58.247777 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8819f6-2214-40da-9ead-8582257d07f9 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:58.248279 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8819f6-2214-40da-9ead-8582257d07f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:58.248759 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8819f6-2214-40da-9ead-8582257d07f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:58.249615 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8819f6-2214-40da-9ead-8582257d07f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:58.250354 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8819f6-2214-40da-9ead-8582257d07f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:58.252092 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8819f6-2214-40da-9ead-8582257d07f9 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:58.269092 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 466/920] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:58.274407 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8819f6-2214-40da-9ead-8582257d07f9 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:58.281651 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:58.293293 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73732ed-2285-42e7-9edf-a3deae4c5e10 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:58.329960 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45161df-e181-44e2-b7f3-770e35a7d04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:58.331009 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45161df-e181-44e2-b7f3-770e35a7d04f None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:58.331531 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45161df-e181-44e2-b7f3-770e35a7d04f None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:58.332018 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45161df-e181-44e2-b7f3-770e35a7d04f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:58.332789 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45161df-e181-44e2-b7f3-770e35a7d04f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:58.334372 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45161df-e181-44e2-b7f3-770e35a7d04f None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:58.343693 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-e45161df-e181-44e2-b7f3-770e35a7d04f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 14:13:58.343693 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:58.343693 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:13:58.343693 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:13:58.343693 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:13:58.343693 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:13:58.343693 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:13:58.343693 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:13:58.343693 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 14:13:58.343693 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:58.343693 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:13:58.343693 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:58.343693 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:58.343693 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:58.343693 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:58.343693 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:58.343693 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:58.344674 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:58.344674 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:58.344674 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:58.344674 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:58.344674 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:58.344674 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:58.344674 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:13:58.344674 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:13:58.344674 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:13:58.344674 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:13:58.344674 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:58.344674 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:58.344674 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:58.344674 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:58.344674 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:58.344674 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:58.345484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:58.345484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:58.345484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:58.345484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:58.345484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:58.345484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:58.345484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:58.345484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:58.345484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:58.345484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:58.345484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:13:58.345484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:13:58.345484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 14:13:58.345484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:58.348200 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 455/921] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 14:13:58 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:13:58.354700 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45161df-e181-44e2-b7f3-770e35a7d04f None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:58.357901 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:58.402666 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8819f6-2214-40da-9ead-8582257d07f9 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:58.440664 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71479d-2d6b-4eb9-b0af-11fe4a24f2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:58.441625 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71479d-2d6b-4eb9-b0af-11fe4a24f2a2 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:58.442116 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71479d-2d6b-4eb9-b0af-11fe4a24f2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:58.442670 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71479d-2d6b-4eb9-b0af-11fe4a24f2a2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:58.453064 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e71479d-2d6b-4eb9-b0af-11fe4a24f2a2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:58.455016 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e71479d-2d6b-4eb9-b0af-11fe4a24f2a2 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:58.468747 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-2e71479d-2d6b-4eb9-b0af-11fe4a24f2a2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 14:13:58.468747 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:58.468747 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:58.468747 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:58.468747 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:58.468747 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:58.468747 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:58.468747 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:58.468747 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:58.468747 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:58.468747 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:58.468747 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:58.468747 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:13:58.468747 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:13:58.468747 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:13:58.468747 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:13:58.469661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:58.469661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:58.469661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:13:58.469661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:58.469661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:13:58.469661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:13:58.469661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:58.469661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:58.469661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:58.469661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:58.469661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:58.469661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:58.469661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:58.469661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:58.469661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:58.469661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:58.469661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:58.469661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:58.470584 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:58.470584 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:58.470584 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:13:58.470584 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:13:58.470584 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:13:58.470584 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:13:58.470584 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:13:58.470584 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:13:58.470584 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 14:13:58.470584 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:58.473072 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 467/922] 10.222.0.52 () {56 vars in 1117 bytes} [Sat Sep 24 14:13:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:13:58.479533 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71479d-2d6b-4eb9-b0af-11fe4a24f2a2 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:58.485170 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45161df-e181-44e2-b7f3-770e35a7d04f None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:58.521487 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a88906-3888-4050-abc5-510f70a6a2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:58.522576 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a88906-3888-4050-abc5-510f70a6a2ee None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:58.523139 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a88906-3888-4050-abc5-510f70a6a2ee None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:58.523604 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a88906-3888-4050-abc5-510f70a6a2ee None admin] PATH_INFO: `/v3/services` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:58.525069 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a88906-3888-4050-abc5-510f70a6a2ee None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:58.525561 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a88906-3888-4050-abc5-510f70a6a2ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:58.527238 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a88906-3888-4050-abc5-510f70a6a2ee None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:58.538028 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 456/923] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 14:13:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:13:58.539133 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a88906-3888-4050-abc5-510f70a6a2ee None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:58.572083 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:58.607188 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71479d-2d6b-4eb9-b0af-11fe4a24f2a2 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:58.633046 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:58.645901 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78049dbe-4a9f-41b0-88ca-0673294610e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:58.646953 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78049dbe-4a9f-41b0-88ca-0673294610e0 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:58.647428 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78049dbe-4a9f-41b0-88ca-0673294610e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:58.647925 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78049dbe-4a9f-41b0-88ca-0673294610e0 None admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:58.648965 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78049dbe-4a9f-41b0-88ca-0673294610e0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:58.649950 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78049dbe-4a9f-41b0-88ca-0673294610e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:58.651644 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78049dbe-4a9f-41b0-88ca-0673294610e0 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:58.672648 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 468/924] 10.222.0.52 () {56 vars in 1149 bytes} [Sat Sep 24 14:13:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:13:58.678619 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78049dbe-4a9f-41b0-88ca-0673294610e0 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:58.690043 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a88906-3888-4050-abc5-510f70a6a2ee None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:58.726765 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c78af0-0b5c-4642-9066-d963e6e57372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:58.727834 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c78af0-0b5c-4642-9066-d963e6e57372 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:58.728304 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c78af0-0b5c-4642-9066-d963e6e57372 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:58.728813 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c78af0-0b5c-4642-9066-d963e6e57372 None admin] PATH_INFO: `/v3/services` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:58.729745 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c78af0-0b5c-4642-9066-d963e6e57372 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:58.730237 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c78af0-0b5c-4642-9066-d963e6e57372 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:58.731821 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c78af0-0b5c-4642-9066-d963e6e57372 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:58.742159 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 457/925] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 14:13:58 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:58.743640 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c78af0-0b5c-4642-9066-d963e6e57372 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:58.792605 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78049dbe-4a9f-41b0-88ca-0673294610e0 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:58.828721 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99760175-294f-4c94-be5d-2ec20de02ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:58.829347 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99760175-294f-4c94-be5d-2ec20de02ef7 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:58.832503 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99760175-294f-4c94-be5d-2ec20de02ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:58.832503 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99760175-294f-4c94-be5d-2ec20de02ef7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:58.841267 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99760175-294f-4c94-be5d-2ec20de02ef7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:58.842737 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99760175-294f-4c94-be5d-2ec20de02ef7 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:58.853096 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:58.859024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-99760175-294f-4c94-be5d-2ec20de02ef7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:58.859024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:58.859024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:58.859024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:58.859024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:58.859024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:58.859024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:58.859024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:58.859024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:58.859024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:58.859024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:58.859024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:58.859024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:13:58.859024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:13:58.859024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:13:58.859024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:13:58.860391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:58.860391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:58.860391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:58.860391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:58.860391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:58.860391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:58.860391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:58.860391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:58.860391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:58.860391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:58.860391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:58.860391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:58.860391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:58.860391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:58.860391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:58.860391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:58.860391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:13:58.861269 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:13:58.861269 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:13:58.861269 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:58.862927 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 469/926] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:13:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:13:58.865262 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99760175-294f-4c94-be5d-2ec20de02ef7 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:58.877395 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c78af0-0b5c-4642-9066-d963e6e57372 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:58.904697 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:58.917187 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b4d35f-60fb-48e7-b3be-a9ca2977f3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:58.917793 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4d35f-60fb-48e7-b3be-a9ca2977f3ee None admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:58.918327 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4d35f-60fb-48e7-b3be-a9ca2977f3ee None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:58.918327 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4d35f-60fb-48e7-b3be-a9ca2977f3ee None admin] PATH_INFO: `/v3/endpoints` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:58.919017 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b4d35f-60fb-48e7-b3be-a9ca2977f3ee None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:58.920362 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b4d35f-60fb-48e7-b3be-a9ca2977f3ee None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:58.950844 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 458/927] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:13:58 2022] POST /identity/v3/endpoints => generated 346 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:13:58.950844 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3e940-95a8-428b-9d02-47dfbd70accc None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:58.950844 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3e940-95a8-428b-9d02-47dfbd70accc None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:58.950844 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3e940-95a8-428b-9d02-47dfbd70accc None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:58.950844 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 459/928] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:13:58.958081 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5115e46b-5f52-43ef-9b88-a96cb92924f2 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:58.958418 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5115e46b-5f52-43ef-9b88-a96cb92924f2 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:58.959012 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5115e46b-5f52-43ef-9b88-a96cb92924f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:58.981379 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-5115e46b-5f52-43ef-9b88-a96cb92924f2 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:58.982168 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99760175-294f-4c94-be5d-2ec20de02ef7 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:58.982585 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-5115e46b-5f52-43ef-9b88-a96cb92924f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:59.031434 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59bcd9-35df-4fa6-9bbd-052f5b3dc964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:59.032956 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59bcd9-35df-4fa6-9bbd-052f5b3dc964 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:59.033556 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59bcd9-35df-4fa6-9bbd-052f5b3dc964 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:59.033829 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59bcd9-35df-4fa6-9bbd-052f5b3dc964 None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:59.034508 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb59bcd9-35df-4fa6-9bbd-052f5b3dc964 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:59.034928 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb59bcd9-35df-4fa6-9bbd-052f5b3dc964 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:59.036352 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb59bcd9-35df-4fa6-9bbd-052f5b3dc964 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:59.055004 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 470/929] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:59.064089 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59bcd9-35df-4fa6-9bbd-052f5b3dc964 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:59.078867 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 460/930] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:13:59.180084 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:59.191052 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59bcd9-35df-4fa6-9bbd-052f5b3dc964 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:59.226860 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954ef0aa-1e33-423d-ae8f-45c9d994f19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:59.227798 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ef0aa-1e33-423d-ae8f-45c9d994f19a None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:59.228214 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ef0aa-1e33-423d-ae8f-45c9d994f19a None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:59.228619 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ef0aa-1e33-423d-ae8f-45c9d994f19a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:59.235725 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954ef0aa-1e33-423d-ae8f-45c9d994f19a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:59.236343 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6030927-848f-480a-b97c-9c4a7ab93068 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:59.236633 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6030927-848f-480a-b97c-9c4a7ab93068 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:59.236914 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6030927-848f-480a-b97c-9c4a7ab93068 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:59.237802 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954ef0aa-1e33-423d-ae8f-45c9d994f19a None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:59.238463 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 461/931] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:13:59.243971 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1051e2-2d0b-4606-892b-ca546f563903 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:59.244254 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1051e2-2d0b-4606-892b-ca546f563903 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:59.244525 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1051e2-2d0b-4606-892b-ca546f563903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:59.251081 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-954ef0aa-1e33-423d-ae8f-45c9d994f19a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:59.251081 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:59.251081 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:59.251081 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:59.251081 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:59.251081 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:59.251081 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:59.251081 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:59.251081 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:59.251081 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:59.251081 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:59.251081 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:59.251081 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:13:59.251081 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:13:59.251081 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:13:59.252419 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:13:59.252419 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:59.252419 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:59.252419 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:59.252419 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:59.252419 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:59.252419 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:59.252419 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:59.252419 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:59.252419 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:59.252419 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:59.252419 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:59.252419 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:59.252419 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:59.252419 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:59.252419 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:59.252419 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:59.253595 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:13:59.253595 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:13:59.253595 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:13:59.253595 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:13:59.253595 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:13:59.253595 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:13:59.253595 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:13:59.253595 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:59.255654 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 471/932] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:13:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:13:59.263724 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954ef0aa-1e33-423d-ae8f-45c9d994f19a None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:59.265434 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-ec1051e2-2d0b-4606-892b-ca546f563903 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:13:59.266780 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-ec1051e2-2d0b-4606-892b-ca546f563903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:13:59.337299 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:59.345276 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 462/933] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:13:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:13:59.377628 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1051e2-2d0b-4606-892b-ca546f563903 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:59.384284 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954ef0aa-1e33-423d-ae8f-45c9d994f19a None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:59.430211 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b241935d-16ed-4b28-bbaf-acd73cf5f069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:59.431439 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b241935d-16ed-4b28-bbaf-acd73cf5f069 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:59.432359 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b241935d-16ed-4b28-bbaf-acd73cf5f069 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:59.432808 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b241935d-16ed-4b28-bbaf-acd73cf5f069 None admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:59.433919 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b241935d-16ed-4b28-bbaf-acd73cf5f069 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:59.434716 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b241935d-16ed-4b28-bbaf-acd73cf5f069 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:59.436750 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b241935d-16ed-4b28-bbaf-acd73cf5f069 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:59.455303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 472/934] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 14:13:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:13:59.461713 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b241935d-16ed-4b28-bbaf-acd73cf5f069 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:59.478263 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:59.496098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1051e2-2d0b-4606-892b-ca546f563903 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:59.536675 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c6ef9-43ad-481f-834d-71270cb3fd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:59.537306 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c6ef9-43ad-481f-834d-71270cb3fd8d None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:59.537581 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c6ef9-43ad-481f-834d-71270cb3fd8d None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:59.537855 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c6ef9-43ad-481f-834d-71270cb3fd8d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:59.538435 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8c6ef9-43ad-481f-834d-71270cb3fd8d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:59.539817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8c6ef9-43ad-481f-834d-71270cb3fd8d None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:59.549611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-ca8c6ef9-43ad-481f-834d-71270cb3fd8d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 14:13:59.549611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:59.549611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:13:59.549611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:13:59.549611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:13:59.549611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:13:59.549611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:13:59.549611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:13:59.549611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 14:13:59.549611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:59.549611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:13:59.549611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:59.549611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:59.549611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:59.549611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:59.549611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:59.549611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:59.555599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:59.555599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:59.555599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:59.555599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:59.555599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:59.555599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:59.555599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:13:59.555599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:13:59.555599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:13:59.555599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:13:59.555599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:59.555599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:59.555599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:59.555599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:59.555599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:59.555599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:59.556503 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:59.556503 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:59.556503 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:59.556503 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:59.556503 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:59.556503 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:59.556503 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:59.556503 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:59.556503 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:59.556503 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:59.556503 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:13:59.556503 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:13:59.556503 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 14:13:59.556503 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:13:59.556503 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 463/935] 10.222.0.52 () {56 vars in 1126 bytes} [Sat Sep 24 14:13:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:13:59.559614 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c6ef9-43ad-481f-834d-71270cb3fd8d None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:59.589537 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b241935d-16ed-4b28-bbaf-acd73cf5f069 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:59.631453 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59186e7-0e77-4377-9398-7111eae9a051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:59.632116 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59186e7-0e77-4377-9398-7111eae9a051 None admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:59.632388 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59186e7-0e77-4377-9398-7111eae9a051 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:59.632655 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59186e7-0e77-4377-9398-7111eae9a051 None admin] PATH_INFO: `/v3/projects/36a3ae7bab25471fb989136150b6438c/users/82328b7fd50649a485c6eaf27c5597c9/roles/30791f3f2a7841d2a80aab92cf4f23f5` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:59.634550 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:59.634834 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59186e7-0e77-4377-9398-7111eae9a051 None admin] RBAC: Authorizing `identity:create_grant(project_id=36a3ae7bab25471fb989136150b6438c, user_id=82328b7fd50649a485c6eaf27c5597c9, role_id=30791f3f2a7841d2a80aab92cf4f23f5)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:59.636958 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59186e7-0e77-4377-9398-7111eae9a051 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:59.658844 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 473/936] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 14:13:59 2022] PUT /identity/v3/projects/36a3ae7bab25471fb989136150b6438c/users/82328b7fd50649a485c6eaf27c5597c9/roles/30791f3f2a7841d2a80aab92cf4f23f5 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:13:59.658844 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59186e7-0e77-4377-9398-7111eae9a051 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:59.695142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c6ef9-43ad-481f-834d-71270cb3fd8d None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:59.737930 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0756fba0-cb9d-453d-9e5d-f1695af4122e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:59.738659 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756fba0-cb9d-453d-9e5d-f1695af4122e None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:59.738964 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756fba0-cb9d-453d-9e5d-f1695af4122e None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:59.739256 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756fba0-cb9d-453d-9e5d-f1695af4122e None admin] PATH_INFO: `/v3/services` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:59.739976 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0756fba0-cb9d-453d-9e5d-f1695af4122e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:59.740278 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0756fba0-cb9d-453d-9e5d-f1695af4122e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:59.741661 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0756fba0-cb9d-453d-9e5d-f1695af4122e None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:59.748838 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 464/937] 10.222.0.52 () {56 vars in 1122 bytes} [Sat Sep 24 14:13:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:13:59.754908 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0756fba0-cb9d-453d-9e5d-f1695af4122e None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:59.757338 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:59.807076 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59186e7-0e77-4377-9398-7111eae9a051 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:59.858447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4e4e9-2813-4cab-bd4d-507b0081d712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:59.859282 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4e4e9-2813-4cab-bd4d-507b0081d712 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:59.859658 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4e4e9-2813-4cab-bd4d-507b0081d712 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:59.860175 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4e4e9-2813-4cab-bd4d-507b0081d712 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:59.860640 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce4e4e9-2813-4cab-bd4d-507b0081d712 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:59.862089 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce4e4e9-2813-4cab-bd4d-507b0081d712 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:59.872888 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-4ce4e4e9-2813-4cab-bd4d-507b0081d712 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 14:13:59.872888 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:59.872888 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:13:59.872888 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:13:59.872888 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:13:59.872888 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:13:59.872888 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:13:59.872888 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:13:59.872888 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 14:13:59.872888 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:59.872888 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:13:59.872888 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:59.872888 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:13:59.872888 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:13:59.872888 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:13:59.872888 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:13:59.874032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:13:59.874032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:13:59.874032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:13:59.874032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:13:59.874032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:13:59.874032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:13:59.874032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:13:59.874032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:13:59.874032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:13:59.874032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:13:59.874032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:13:59.874032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:13:59.874032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:13:59.874032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:13:59.874032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:13:59.874032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:13:59.874032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:13:59.875064 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:13:59.875064 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:13:59.875064 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:13:59.875064 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:13:59.875064 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:13:59.875064 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:13:59.875064 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:13:59.875064 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:13:59.875064 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:13:59.875064 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:13:59.875064 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:13:59.875064 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:13:59.875064 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 14:13:59.875064 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:13:59.875064 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 474/938] 10.222.0.52 () {56 vars in 1132 bytes} [Sat Sep 24 14:13:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:13:59.880473 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4e4e9-2813-4cab-bd4d-507b0081d712 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:13:59.896710 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0756fba0-cb9d-453d-9e5d-f1695af4122e None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:13:59.944658 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:13:59.956569 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f349989b-9fc7-4742-8c79-1c12a57cc552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:13:59.957469 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349989b-9fc7-4742-8c79-1c12a57cc552 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:13:59.957893 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349989b-9fc7-4742-8c79-1c12a57cc552 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:13:59.958410 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349989b-9fc7-4742-8c79-1c12a57cc552 None admin] PATH_INFO: `/v3/services` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:13:59.959289 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f349989b-9fc7-4742-8c79-1c12a57cc552 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:13:59.959772 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f349989b-9fc7-4742-8c79-1c12a57cc552 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:13:59.962506 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f349989b-9fc7-4742-8c79-1c12a57cc552 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:13:59.968327 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 465/939] 10.222.0.52 () {56 vars in 1122 bytes} [Sat Sep 24 14:13:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:13:59.974240 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f349989b-9fc7-4742-8c79-1c12a57cc552 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:00.012280 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4e4e9-2813-4cab-bd4d-507b0081d712 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:00.055270 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787fda67-3d3b-4c79-87e1-18e849522eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:00.056827 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787fda67-3d3b-4c79-87e1-18e849522eab None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:00.057173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787fda67-3d3b-4c79-87e1-18e849522eab None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:00.057519 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787fda67-3d3b-4c79-87e1-18e849522eab None admin] PATH_INFO: `/v3/services` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:00.058265 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787fda67-3d3b-4c79-87e1-18e849522eab None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:00.058880 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787fda67-3d3b-4c79-87e1-18e849522eab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:00.060357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787fda67-3d3b-4c79-87e1-18e849522eab None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:00.061674 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:00.074565 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 475/940] 10.222.0.52 () {56 vars in 1126 bytes} [Sat Sep 24 14:13:59 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:00.076231 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787fda67-3d3b-4c79-87e1-18e849522eab None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:00.121163 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f349989b-9fc7-4742-8c79-1c12a57cc552 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:00.167745 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22981d1-427d-4b3c-956b-10793d7e010c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:00.168391 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22981d1-427d-4b3c-956b-10793d7e010c None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:00.168674 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22981d1-427d-4b3c-956b-10793d7e010c None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:00.168946 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22981d1-427d-4b3c-956b-10793d7e010c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:00.169652 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22981d1-427d-4b3c-956b-10793d7e010c None admin] RBAC: Adding query filter params (interface=public, service_id=678c93a260fa4a66b1cf09f60228b78f, region_id=RegionOne) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:00.169958 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22981d1-427d-4b3c-956b-10793d7e010c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:00.171370 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22981d1-427d-4b3c-956b-10793d7e010c None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:00.182340 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 466/941] 10.222.0.52 () {56 vars in 1251 bytes} [Sat Sep 24 14:13:59 2022] GET /identity/v3/endpoints?service_id=678c93a260fa4a66b1cf09f60228b78f&interface=public®ion_id=RegionOne => generated 194 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:00.183744 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22981d1-427d-4b3c-956b-10793d7e010c None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:00.207720 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787fda67-3d3b-4c79-87e1-18e849522eab None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:00.249309 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:00.249694 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a603c-08d9-444e-bc18-11b2a79a630b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:00.250447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a603c-08d9-444e-bc18-11b2a79a630b None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:00.250560 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a603c-08d9-444e-bc18-11b2a79a630b None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:00.250866 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a603c-08d9-444e-bc18-11b2a79a630b None admin] PATH_INFO: `/v3/services` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:00.251667 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0a603c-08d9-444e-bc18-11b2a79a630b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:00.252007 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0a603c-08d9-444e-bc18-11b2a79a630b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:00.254106 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0a603c-08d9-444e-bc18-11b2a79a630b None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:00.266308 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 476/942] 10.222.0.52 () {56 vars in 1126 bytes} [Sat Sep 24 14:14:00 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:14:00.267709 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a603c-08d9-444e-bc18-11b2a79a630b None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:00.304925 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22981d1-427d-4b3c-956b-10793d7e010c None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:00.339206 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:00.342651 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc41c7de-3069-4093-ba88-8ff0f6738a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:00.343298 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc41c7de-3069-4093-ba88-8ff0f6738a3b None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:00.343580 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc41c7de-3069-4093-ba88-8ff0f6738a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:00.343853 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc41c7de-3069-4093-ba88-8ff0f6738a3b None admin] PATH_INFO: `/v3/services` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:00.344460 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc41c7de-3069-4093-ba88-8ff0f6738a3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:00.346761 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc41c7de-3069-4093-ba88-8ff0f6738a3b None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:00.358850 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 467/943] 10.222.0.52 () {56 vars in 1087 bytes} [Sat Sep 24 14:14:00 2022] GET /identity/v3/services => generated 1497 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 14:14:00.385326 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a603c-08d9-444e-bc18-11b2a79a630b None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:00.425968 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d5c8b6-5850-4b17-9ea1-ae8f4e648488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:00.426598 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5c8b6-5850-4b17-9ea1-ae8f4e648488 None admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:00.426919 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5c8b6-5850-4b17-9ea1-ae8f4e648488 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:00.427197 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5c8b6-5850-4b17-9ea1-ae8f4e648488 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:00.427878 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d5c8b6-5850-4b17-9ea1-ae8f4e648488 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:00.429276 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d5c8b6-5850-4b17-9ea1-ae8f4e648488 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:00.455048 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 477/944] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:14:00 2022] POST /identity/v3/endpoints => generated 347 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:14:01.240478 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c434d-4fae-4f0f-9559-8f75b2e1d9a4 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:01.241030 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c434d-4fae-4f0f-9559-8f75b2e1d9a4 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:01.241030 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c434d-4fae-4f0f-9559-8f75b2e1d9a4 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:01.242374 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 468/945] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:14:01.248692 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bb220-175b-4252-98d6-7ee3d093862c None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:01.249147 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bb220-175b-4252-98d6-7ee3d093862c None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:01.249228 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bb220-175b-4252-98d6-7ee3d093862c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:01.268166 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-4e0bb220-175b-4252-98d6-7ee3d093862c None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:14:01.269359 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-4e0bb220-175b-4252-98d6-7ee3d093862c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:14:01.351773 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 478/946] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:14:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:14:01.625094 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c434d-4fae-4f0f-9559-8f75b2e1d9a4 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:01.731816 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c434d-4fae-4f0f-9559-8f75b2e1d9a4 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:01.749828 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:01.768041 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf7254-20ad-43ea-86e3-67e1ddebb192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:01.768644 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf7254-20ad-43ea-86e3-67e1ddebb192 None admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:01.768920 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf7254-20ad-43ea-86e3-67e1ddebb192 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:01.769185 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf7254-20ad-43ea-86e3-67e1ddebb192 None admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:01.769910 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdf7254-20ad-43ea-86e3-67e1ddebb192 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:01.771408 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdf7254-20ad-43ea-86e3-67e1ddebb192 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:01.774015 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: WARNING keystone.server.flask.common [None req-cbdf7254-20ad-43ea-86e3-67e1ddebb192 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 24 14:14:01.797282 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.sql.core [None req-cbdf7254-20ad-43ea-86e3-67e1ddebb192 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 24 14:14:01.797282 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 24 14:14:01.797282 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [parameters: {'id': 'f2ae2e4234534325b41eb82751f626d3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 24 14:14:01.797282 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81497) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 24 14:14:01.815705 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-cbdf7254-20ad-43ea-86e3-67e1ddebb192 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f2ae2e4234534325b41eb82751f626d3.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f2ae2e4234534325b41eb82751f626d3. Sep 24 14:14:01.815705 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:01.815705 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 24 14:14:01.815705 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 24 14:14:01.815705 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 24 14:14:01.815705 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 24 14:14:01.815705 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 24 14:14:01.815705 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application result = self._query(query) Sep 24 14:14:01.815705 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 24 14:14:01.815705 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application conn.query(q) Sep 24 14:14:01.815705 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 24 14:14:01.815705 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 24 14:14:01.815705 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 24 14:14:01.816618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application result.read() Sep 24 14:14:01.816618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 24 14:14:01.816618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 24 14:14:01.816618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 24 14:14:01.816618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 24 14:14:01.816618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 24 14:14:01.816618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 24 14:14:01.816618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 24 14:14:01.816618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 24 14:14:01.816618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 24 14:14:01.816618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:01.816618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 24 14:14:01.816618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:01.816618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:01.816618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 24 14:14:01.816618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 24 14:14:01.816618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 24 14:14:01.816618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 24 14:14:01.817577 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 24 14:14:01.817577 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application next(self.gen) Sep 24 14:14:01.817577 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 24 14:14:01.817577 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application yield resource Sep 24 14:14:01.817577 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 24 14:14:01.817577 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application next(self.gen) Sep 24 14:14:01.817577 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 24 14:14:01.817577 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application self.session.rollback() Sep 24 14:14:01.817577 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 24 14:14:01.817577 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application self.force_reraise() Sep 24 14:14:01.817577 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 24 14:14:01.817577 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise self.value Sep 24 14:14:01.817577 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 24 14:14:01.817577 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 24 14:14:01.817577 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 24 14:14:01.817577 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application session.commit() Sep 24 14:14:01.817577 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 24 14:14:01.818469 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 24 14:14:01.818469 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 24 14:14:01.818469 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application self._prepare_impl() Sep 24 14:14:01.818469 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 24 14:14:01.818469 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application self.session.flush() Sep 24 14:14:01.818469 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 24 14:14:01.818469 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application self._flush(objects) Sep 24 14:14:01.818469 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 24 14:14:01.818469 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 24 14:14:01.818469 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 24 14:14:01.818469 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application compat.raise_( Sep 24 14:14:01.818469 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 24 14:14:01.818469 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception Sep 24 14:14:01.818469 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 24 14:14:01.818469 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application flush_context.execute() Sep 24 14:14:01.818469 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 24 14:14:01.818469 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rec.execute(self) Sep 24 14:14:01.819484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 24 14:14:01.819484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 24 14:14:01.819484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 24 14:14:01.819484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 24 14:14:01.819484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 24 14:14:01.819484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 24 14:14:01.819484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 24 14:14:01.819484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 24 14:14:01.819484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 24 14:14:01.819484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 24 14:14:01.819484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 24 14:14:01.819484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 24 14:14:01.819484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 24 14:14:01.819484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 24 14:14:01.819484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 24 14:14:01.820294 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 24 14:14:01.820294 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 24 14:14:01.820294 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception Sep 24 14:14:01.820294 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 24 14:14:01.820294 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 24 14:14:01.820294 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 24 14:14:01.820294 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 24 14:14:01.820294 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 24 14:14:01.820294 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application result = self._query(query) Sep 24 14:14:01.820294 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 24 14:14:01.820294 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application conn.query(q) Sep 24 14:14:01.820294 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 24 14:14:01.820294 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 24 14:14:01.820294 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 24 14:14:01.820294 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application result.read() Sep 24 14:14:01.820294 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 24 14:14:01.820294 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 24 14:14:01.821206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 24 14:14:01.821206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 24 14:14:01.821206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 24 14:14:01.821206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 24 14:14:01.821206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 24 14:14:01.821206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 24 14:14:01.821206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 24 14:14:01.821206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 24 14:14:01.821206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [parameters: {'id': 'f2ae2e4234534325b41eb82751f626d3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 24 14:14:01.821206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 24 14:14:01.821206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:01.821206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:14:01.821206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:01.821206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:01.821206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 24 14:14:01.822004 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 24 14:14:01.822004 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 24 14:14:01.822004 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 24 14:14:01.822004 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 24 14:14:01.822004 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:01.822004 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:14:01.822004 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:01.822004 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:01.822004 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:01.822004 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:01.822004 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:01.822004 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:01.822004 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:01.822004 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:01.822004 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:01.822004 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:01.822004 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:01.828466 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:01.828466 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 24 14:14:01.828466 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 24 14:14:01.828466 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:01.828466 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:01.828466 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 24 14:14:01.828466 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 24 14:14:01.828466 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f2ae2e4234534325b41eb82751f626d3. Sep 24 14:14:01.828466 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:01.828466 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 469/947] 10.222.0.52 () {60 vars in 1140 bytes} [Sat Sep 24 14:14:01 2022] POST /identity/v3/projects => generated 262 bytes in 193 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:14:01.829776 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0bb220-175b-4252-98d6-7ee3d093862c None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:01.871768 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:01.921359 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4b323-eec9-499b-83fc-30a97871a4fb None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:01.921678 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4b323-eec9-499b-83fc-30a97871a4fb None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:01.921908 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4b323-eec9-499b-83fc-30a97871a4fb None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:01.928199 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 470/948] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:14:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:14:01.939055 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25bfdd-e896-42d8-9459-b751295c11d2 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:01.939343 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25bfdd-e896-42d8-9459-b751295c11d2 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:01.939617 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25bfdd-e896-42d8-9459-b751295c11d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:01.941065 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0bb220-175b-4252-98d6-7ee3d093862c None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:01.964004 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-fa25bfdd-e896-42d8-9459-b751295c11d2 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:14:01.966196 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-fa25bfdd-e896-42d8-9459-b751295c11d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:14:01.976593 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79d8d3-7d67-441e-9a14-904af7dde49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:01.977999 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79d8d3-7d67-441e-9a14-904af7dde49c None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:01.978471 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79d8d3-7d67-441e-9a14-904af7dde49c None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:01.978966 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79d8d3-7d67-441e-9a14-904af7dde49c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:01.985981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef79d8d3-7d67-441e-9a14-904af7dde49c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:01.987717 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef79d8d3-7d67-441e-9a14-904af7dde49c None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:02.000070 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-ef79d8d3-7d67-441e-9a14-904af7dde49c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:14:02.000070 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:02.000070 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:02.000070 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:02.000070 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:02.000070 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:02.000070 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:02.000070 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:02.000070 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:02.000070 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:02.000070 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:02.000070 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:02.000070 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:14:02.000070 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:14:02.000070 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:14:02.000941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:14:02.000941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:02.000941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:02.000941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:14:02.000941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:14:02.000941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:14:02.000941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:14:02.000941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:14:02.000941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:14:02.000941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:14:02.000941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:14:02.000941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:14:02.000941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:14:02.000941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:14:02.000941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:14:02.000941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:14:02.000941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:14:02.001829 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:14:02.001829 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:14:02.001829 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:14:02.001829 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:14:02.001829 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:14:02.001829 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:14:02.001829 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:14:02.001829 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:14:02.004193 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 479/949] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:14:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:14:02.014535 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79d8d3-7d67-441e-9a14-904af7dde49c None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:02.051102 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 471/950] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:14:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:14:02.131055 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79d8d3-7d67-441e-9a14-904af7dde49c None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:02.133409 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:02.167648 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62cc593-e64d-4561-9ede-f8edfd042dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:02.168259 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62cc593-e64d-4561-9ede-f8edfd042dcc None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:02.168537 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62cc593-e64d-4561-9ede-f8edfd042dcc None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:02.168806 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62cc593-e64d-4561-9ede-f8edfd042dcc None admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:02.169464 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62cc593-e64d-4561-9ede-f8edfd042dcc None admin] RBAC: Adding query filter params (name=service) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:02.169762 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62cc593-e64d-4561-9ede-f8edfd042dcc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:02.171221 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62cc593-e64d-4561-9ede-f8edfd042dcc None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:02.189776 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 480/951] 10.222.0.52 () {56 vars in 1112 bytes} [Sat Sep 24 14:14:02 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:02.330111 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa25bfdd-e896-42d8-9459-b751295c11d2 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:02.440003 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa25bfdd-e896-42d8-9459-b751295c11d2 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:02.442378 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:02.478296 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed1f52-425d-492f-b30c-d90acdc94b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:02.478963 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed1f52-425d-492f-b30c-d90acdc94b69 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:02.479245 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed1f52-425d-492f-b30c-d90acdc94b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:02.479573 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed1f52-425d-492f-b30c-d90acdc94b69 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:02.490261 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ed1f52-425d-492f-b30c-d90acdc94b69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:14:02.495236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-36ed1f52-425d-492f-b30c-d90acdc94b69 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 14:14:02.495236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:02.495236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:02.495236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:02.495236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:02.495236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:02.495236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:02.495236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:02.495236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:02.495236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:02.495236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:02.495236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:02.495236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:14:02.495236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:14:02.495236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:14:02.495236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:14:02.496148 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:14:02.496148 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:14:02.496148 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:14:02.496148 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:14:02.496148 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:02.496148 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:02.496148 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:14:02.496148 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:14:02.496148 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:14:02.496148 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:14:02.496148 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:14:02.496148 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:14:02.496148 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:14:02.496148 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:14:02.496148 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:14:02.496148 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:14:02.496988 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:14:02.496988 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:14:02.496988 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:14:02.496988 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:14:02.496988 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:14:02.496988 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:14:02.496988 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:14:02.496988 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:14:02.496988 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:14:02.496988 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:14:02.496988 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 14:14:02.496988 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:02.499571 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 472/952] 10.222.0.52 () {56 vars in 1120 bytes} [Sat Sep 24 14:14:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:14:02.502738 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62cc593-e64d-4561-9ede-f8edfd042dcc None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:02.623799 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62cc593-e64d-4561-9ede-f8edfd042dcc None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:02.660858 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b592ed-60ad-43f9-94a1-e6549621c294 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:02.661180 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b592ed-60ad-43f9-94a1-e6549621c294 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:02.661394 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b592ed-60ad-43f9-94a1-e6549621c294 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:02.661785 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3ee1d-4c44-4d4f-9e6c-69173a82c07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:02.662644 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3ee1d-4c44-4d4f-9e6c-69173a82c07b None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:02.663082 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 473/953] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:14:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:14:02.663358 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3ee1d-4c44-4d4f-9e6c-69173a82c07b None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:02.663845 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3ee1d-4c44-4d4f-9e6c-69173a82c07b None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:02.664716 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc3ee1d-4c44-4d4f-9e6c-69173a82c07b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:02.665759 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc3ee1d-4c44-4d4f-9e6c-69173a82c07b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:02.667353 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc3ee1d-4c44-4d4f-9e6c-69173a82c07b None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:02.673037 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe3b798-64d2-482b-8df0-1661682eba3e None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:02.673323 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe3b798-64d2-482b-8df0-1661682eba3e None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:02.673600 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe3b798-64d2-482b-8df0-1661682eba3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:02.679057 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 481/954] 10.222.0.52 () {56 vars in 1115 bytes} [Sat Sep 24 14:14:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:02.685304 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3ee1d-4c44-4d4f-9e6c-69173a82c07b None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:02.698040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-dfe3b798-64d2-482b-8df0-1661682eba3e None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:14:02.699517 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-dfe3b798-64d2-482b-8df0-1661682eba3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:14:02.714681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:02.771113 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 474/955] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:14:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:14:02.772797 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd1296-4a2c-484e-9f8b-df00433ae813 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:02.773148 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd1296-4a2c-484e-9f8b-df00433ae813 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:02.773488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd1296-4a2c-484e-9f8b-df00433ae813 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:02.775482 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 475/956] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:14:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:14:02.780733 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ef639-8698-44a6-8458-b38bdfd145a8 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:02.781077 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ef639-8698-44a6-8458-b38bdfd145a8 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:02.781415 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ef639-8698-44a6-8458-b38bdfd145a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:02.800692 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-af2ef639-8698-44a6-8458-b38bdfd145a8 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:14:02.800903 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3ee1d-4c44-4d4f-9e6c-69173a82c07b None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:02.801974 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-af2ef639-8698-44a6-8458-b38bdfd145a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:14:02.841680 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a9c908-d965-4157-9efb-618423c8acc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:02.842418 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9c908-d965-4157-9efb-618423c8acc7 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:02.842762 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9c908-d965-4157-9efb-618423c8acc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:02.843238 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9c908-d965-4157-9efb-618423c8acc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:02.855442 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a9c908-d965-4157-9efb-618423c8acc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:02.857028 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a9c908-d965-4157-9efb-618423c8acc7 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:02.872859 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-36a9c908-d965-4157-9efb-618423c8acc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:14:02.872859 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:02.872859 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:02.872859 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:02.872859 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:02.872859 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:02.872859 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:02.872859 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:02.872859 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:02.872859 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:02.872859 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:02.872859 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:02.872859 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:14:02.872859 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:14:02.872859 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:14:02.872859 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:14:02.874019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:02.874019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:02.874019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:14:02.874019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:14:02.874019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:14:02.874019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:14:02.874019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:14:02.874019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:14:02.874019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:14:02.874019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:14:02.874019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:14:02.874019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:14:02.874019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:14:02.874019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:14:02.874019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:14:02.874019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:14:02.874019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:14:02.875148 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:14:02.875148 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:14:02.875148 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:14:02.875932 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 482/957] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:14:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:14:02.881954 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a9c908-d965-4157-9efb-618423c8acc7 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:02.902856 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 476/958] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:14:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:14:02.987338 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:03.011797 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a9c908-d965-4157-9efb-618423c8acc7 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:03.051034 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed2f62d-3cca-4d7b-b77e-6eea04a47ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:03.051685 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2f62d-3cca-4d7b-b77e-6eea04a47ec5 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:03.051957 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2f62d-3cca-4d7b-b77e-6eea04a47ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:03.052221 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2f62d-3cca-4d7b-b77e-6eea04a47ec5 None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:03.052852 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed2f62d-3cca-4d7b-b77e-6eea04a47ec5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:03.053145 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed2f62d-3cca-4d7b-b77e-6eea04a47ec5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:03.054499 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed2f62d-3cca-4d7b-b77e-6eea04a47ec5 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:03.062414 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ef639-8698-44a6-8458-b38bdfd145a8 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:03.077284 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 483/959] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:14:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:14:03.079997 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed2f62d-3cca-4d7b-b77e-6eea04a47ec5 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:03.098008 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:03.197767 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ef639-8698-44a6-8458-b38bdfd145a8 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:03.210773 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed2f62d-3cca-4d7b-b77e-6eea04a47ec5 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:03.236219 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d6193-afcd-4a73-a1df-add62705b338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:03.236849 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d6193-afcd-4a73-a1df-add62705b338 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:03.237127 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d6193-afcd-4a73-a1df-add62705b338 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:03.237453 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d6193-afcd-4a73-a1df-add62705b338 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:03.237985 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608d6193-afcd-4a73-a1df-add62705b338 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:03.239492 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608d6193-afcd-4a73-a1df-add62705b338 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:03.247599 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e58a1b8-75a5-4142-91dc-596d4882ffe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:03.248228 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e58a1b8-75a5-4142-91dc-596d4882ffe0 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:03.248504 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e58a1b8-75a5-4142-91dc-596d4882ffe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:03.248773 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e58a1b8-75a5-4142-91dc-596d4882ffe0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:03.249040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-608d6193-afcd-4a73-a1df-add62705b338 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 14:14:03.249040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:03.249040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:14:03.249040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:14:03.249040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:14:03.249040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:14:03.249040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:14:03.249040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:14:03.249040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 14:14:03.249040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:03.249040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:14:03.249040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:03.249040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:03.249040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:03.249040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:03.249040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:03.249040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:03.249963 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:03.249963 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:03.249963 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:03.249963 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:03.249963 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:03.249963 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:03.249963 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:14:03.249963 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:14:03.249963 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:14:03.249963 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:14:03.249963 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:03.249963 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:03.249963 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:14:03.249963 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:14:03.249963 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:14:03.249963 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:14:03.250800 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:14:03.250800 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:14:03.250800 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:14:03.250800 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:14:03.250800 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:14:03.250800 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:14:03.250800 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:14:03.250800 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:14:03.250800 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:14:03.250800 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:14:03.250800 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:14:03.250800 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:14:03.250800 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 14:14:03.250800 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:03.250800 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 477/960] 10.222.0.52 () {56 vars in 1126 bytes} [Sat Sep 24 14:14:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:14:03.251642 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d6193-afcd-4a73-a1df-add62705b338 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:03.262313 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e58a1b8-75a5-4142-91dc-596d4882ffe0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:03.264228 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e58a1b8-75a5-4142-91dc-596d4882ffe0 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:03.266584 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:03.277334 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-5e58a1b8-75a5-4142-91dc-596d4882ffe0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 14:14:03.277334 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:03.277334 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:03.277334 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:03.277334 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:03.277334 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:03.277334 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:03.277334 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:03.277334 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:03.277334 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:03.277334 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:03.277334 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:03.277334 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:14:03.277334 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:14:03.277334 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:14:03.277334 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:14:03.278303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:03.278303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:03.278303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:14:03.278303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:14:03.278303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:14:03.278303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:14:03.278303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:14:03.278303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:14:03.278303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:14:03.278303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:14:03.278303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:14:03.278303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:14:03.278303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:14:03.278303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:14:03.278303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:14:03.278303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:14:03.278303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:14:03.278303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:14:03.279775 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:14:03.279775 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:14:03.279775 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:14:03.279775 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:14:03.279775 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:14:03.279775 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:14:03.279775 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:14:03.279775 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:14:03.279775 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 14:14:03.279775 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:14:03.279775 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 484/961] 10.222.0.52 () {56 vars in 1117 bytes} [Sat Sep 24 14:14:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:14:03.279775 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e58a1b8-75a5-4142-91dc-596d4882ffe0 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:03.350029 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:03.363793 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d6193-afcd-4a73-a1df-add62705b338 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:03.387131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e58a1b8-75a5-4142-91dc-596d4882ffe0 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:03.398400 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfda992-ea7f-459c-806e-fea0e3a45667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:03.399047 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfda992-ea7f-459c-806e-fea0e3a45667 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:03.399327 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfda992-ea7f-459c-806e-fea0e3a45667 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:03.399594 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfda992-ea7f-459c-806e-fea0e3a45667 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:03.400161 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfda992-ea7f-459c-806e-fea0e3a45667 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:03.401509 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfda992-ea7f-459c-806e-fea0e3a45667 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:03.410887 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-4bfda992-ea7f-459c-806e-fea0e3a45667 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 14:14:03.410887 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:03.410887 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:14:03.410887 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:14:03.410887 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:14:03.410887 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:14:03.410887 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:14:03.410887 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:14:03.410887 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 14:14:03.410887 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:03.410887 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:14:03.410887 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:03.410887 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:03.410887 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:03.410887 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:03.410887 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:03.410887 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:03.411865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:03.411865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:03.411865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:03.411865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:03.411865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:03.411865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:03.411865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:14:03.411865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:14:03.411865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:14:03.411865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:14:03.411865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:03.411865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:03.411865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:14:03.411865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:14:03.411865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:14:03.411865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:14:03.412704 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:14:03.412704 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:14:03.412704 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:14:03.412704 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:14:03.412704 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:14:03.412704 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:14:03.412704 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:14:03.412704 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:14:03.412704 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:14:03.412704 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:14:03.412704 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:14:03.412704 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:14:03.412704 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 14:14:03.412704 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:03.412704 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 478/962] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:14:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:14:03.413470 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfda992-ea7f-459c-806e-fea0e3a45667 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:03.428297 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f7644-988e-4f61-aa7d-f2e34baafafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:03.428912 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f7644-988e-4f61-aa7d-f2e34baafafb None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:03.429190 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f7644-988e-4f61-aa7d-f2e34baafafb None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:03.429461 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f7644-988e-4f61-aa7d-f2e34baafafb None admin] PATH_INFO: `/v3/services` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:03.430094 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138f7644-988e-4f61-aa7d-f2e34baafafb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:03.430426 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138f7644-988e-4f61-aa7d-f2e34baafafb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:03.432682 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138f7644-988e-4f61-aa7d-f2e34baafafb None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:03.438098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 485/963] 10.222.0.52 () {56 vars in 1122 bytes} [Sat Sep 24 14:14:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:03.438990 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f7644-988e-4f61-aa7d-f2e34baafafb None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:03.505553 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:03.526478 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfda992-ea7f-459c-806e-fea0e3a45667 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:03.552529 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f7644-988e-4f61-aa7d-f2e34baafafb None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:03.564739 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d7917-9cab-4cc2-86e8-ee49a926f8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:03.565343 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d7917-9cab-4cc2-86e8-ee49a926f8dd None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:03.565622 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d7917-9cab-4cc2-86e8-ee49a926f8dd None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:03.565877 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d7917-9cab-4cc2-86e8-ee49a926f8dd None admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:03.566610 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38d7917-9cab-4cc2-86e8-ee49a926f8dd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:03.566940 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38d7917-9cab-4cc2-86e8-ee49a926f8dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:03.568354 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38d7917-9cab-4cc2-86e8-ee49a926f8dd None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:03.585590 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 479/964] 10.222.0.52 () {56 vars in 1149 bytes} [Sat Sep 24 14:14:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:03.586459 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d7917-9cab-4cc2-86e8-ee49a926f8dd None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:03.593699 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346c13b-efe8-4c9f-861b-ba8de0c84556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:03.594591 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346c13b-efe8-4c9f-861b-ba8de0c84556 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:03.595059 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346c13b-efe8-4c9f-861b-ba8de0c84556 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:03.595332 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346c13b-efe8-4c9f-861b-ba8de0c84556 None admin] PATH_INFO: `/v3/services` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:03.595962 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a346c13b-efe8-4c9f-861b-ba8de0c84556 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:03.596260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a346c13b-efe8-4c9f-861b-ba8de0c84556 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:03.597621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a346c13b-efe8-4c9f-861b-ba8de0c84556 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:03.603171 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 486/965] 10.222.0.52 () {56 vars in 1112 bytes} [Sat Sep 24 14:14:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:03.603661 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:03.608658 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346c13b-efe8-4c9f-861b-ba8de0c84556 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:03.696138 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d7917-9cab-4cc2-86e8-ee49a926f8dd None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:03.715744 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346c13b-efe8-4c9f-861b-ba8de0c84556 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:03.733928 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d43803a-9883-4a19-b9e0-873c576f1386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:03.734541 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43803a-9883-4a19-b9e0-873c576f1386 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:03.734845 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43803a-9883-4a19-b9e0-873c576f1386 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:03.735242 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43803a-9883-4a19-b9e0-873c576f1386 None admin] PATH_INFO: `/v3/services` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:03.735870 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d43803a-9883-4a19-b9e0-873c576f1386 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:03.736171 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d43803a-9883-4a19-b9e0-873c576f1386 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:03.737525 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d43803a-9883-4a19-b9e0-873c576f1386 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:03.740725 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:03.748909 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 480/966] 10.222.0.52 () {56 vars in 1122 bytes} [Sat Sep 24 14:14:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:03.748909 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d43803a-9883-4a19-b9e0-873c576f1386 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:03.755805 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053d25ad-b2dc-423f-afa1-656e29b9ede7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:03.756356 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d25ad-b2dc-423f-afa1-656e29b9ede7 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:03.756632 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d25ad-b2dc-423f-afa1-656e29b9ede7 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:03.756901 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d25ad-b2dc-423f-afa1-656e29b9ede7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:03.826082 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:03.835527 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053d25ad-b2dc-423f-afa1-656e29b9ede7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:03.838001 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053d25ad-b2dc-423f-afa1-656e29b9ede7 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:03.855184 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d43803a-9883-4a19-b9e0-873c576f1386 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:03.894622 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b885ce8c-36e4-4cc7-9f06-e4aaca2e2082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:03.895057 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885ce8c-36e4-4cc7-9f06-e4aaca2e2082 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:03.895326 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885ce8c-36e4-4cc7-9f06-e4aaca2e2082 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:03.895597 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885ce8c-36e4-4cc7-9f06-e4aaca2e2082 None admin] PATH_INFO: `/v3/services` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:03.896242 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b885ce8c-36e4-4cc7-9f06-e4aaca2e2082 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:03.896539 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b885ce8c-36e4-4cc7-9f06-e4aaca2e2082 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:03.897880 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b885ce8c-36e4-4cc7-9f06-e4aaca2e2082 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:03.903347 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 481/967] 10.222.0.52 () {56 vars in 1112 bytes} [Sat Sep 24 14:14:03 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:03.903347 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b885ce8c-36e4-4cc7-9f06-e4aaca2e2082 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:03.952574 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 487/968] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:14:03 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 14:14:03.957986 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053d25ad-b2dc-423f-afa1-656e29b9ede7 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:03.989639 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:04.057371 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b885ce8c-36e4-4cc7-9f06-e4aaca2e2082 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:04.095269 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053d25ad-b2dc-423f-afa1-656e29b9ede7 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:04.095747 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d917bdd4-48bd-4a7d-9568-604cf7ac9283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:04.096361 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917bdd4-48bd-4a7d-9568-604cf7ac9283 None admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:04.096636 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917bdd4-48bd-4a7d-9568-604cf7ac9283 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:04.096902 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917bdd4-48bd-4a7d-9568-604cf7ac9283 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:04.097572 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d917bdd4-48bd-4a7d-9568-604cf7ac9283 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:04.098973 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d917bdd4-48bd-4a7d-9568-604cf7ac9283 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:04.109480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:04.127702 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 482/969] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:14:03 2022] POST /identity/v3/endpoints => generated 349 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:14:04.153812 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d7ac63-164c-4a83-b74f-e2aba7e2edfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:04.154928 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7ac63-164c-4a83-b74f-e2aba7e2edfe None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:04.155432 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7ac63-164c-4a83-b74f-e2aba7e2edfe None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:04.155908 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7ac63-164c-4a83-b74f-e2aba7e2edfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:04.169154 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d7ac63-164c-4a83-b74f-e2aba7e2edfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:04.171084 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d7ac63-164c-4a83-b74f-e2aba7e2edfe None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:04.188613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-46d7ac63-164c-4a83-b74f-e2aba7e2edfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:14:04.188613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:04.188613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:04.188613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:04.188613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:04.188613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:04.188613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:04.188613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:04.188613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:04.188613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:04.188613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:04.188613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:04.188613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:14:04.188613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:14:04.188613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:14:04.188613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:14:04.189631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:04.189631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:04.189631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:14:04.189631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:14:04.189631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:14:04.189631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:14:04.189631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:14:04.189631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:14:04.189631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:14:04.189631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:14:04.189631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:14:04.189631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:14:04.189631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:14:04.189631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:14:04.189631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:14:04.189631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:14:04.189631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:14:04.190578 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:14:04.190578 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:14:04.190578 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:14:04.193015 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 488/970] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:14:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:14:04.200034 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d917bdd4-48bd-4a7d-9568-604cf7ac9283 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:04.314902 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d917bdd4-48bd-4a7d-9568-604cf7ac9283 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:04.352461 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1764d4-6d41-419c-b6b6-f5fff119ea51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:04.353874 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1764d4-6d41-419c-b6b6-f5fff119ea51 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:04.354376 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1764d4-6d41-419c-b6b6-f5fff119ea51 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:04.354890 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1764d4-6d41-419c-b6b6-f5fff119ea51 None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:04.355763 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1764d4-6d41-419c-b6b6-f5fff119ea51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:04.356265 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1764d4-6d41-419c-b6b6-f5fff119ea51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:04.357834 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1764d4-6d41-419c-b6b6-f5fff119ea51 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:04.371805 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 483/971] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:14:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:04.378111 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d7ac63-164c-4a83-b74f-e2aba7e2edfe None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:04.462501 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:04.490655 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d7ac63-164c-4a83-b74f-e2aba7e2edfe None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:04.500300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318024b-6fea-40c9-8efb-964111841cf1 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:04.500583 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318024b-6fea-40c9-8efb-964111841cf1 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:04.500850 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318024b-6fea-40c9-8efb-964111841cf1 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:04.502314 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 484/972] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:14:04.507484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a3fa7-33dd-4706-841e-d137a0f365e4 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:04.507931 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a3fa7-33dd-4706-841e-d137a0f365e4 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:04.508347 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a3fa7-33dd-4706-841e-d137a0f365e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:04.528937 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-7e0a3fa7-33dd-4706-841e-d137a0f365e4 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:14:04.530425 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-7e0a3fa7-33dd-4706-841e-d137a0f365e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:14:04.534707 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87030dc6-503c-44a9-9f6a-fcb5ce89fb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:04.535376 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87030dc6-503c-44a9-9f6a-fcb5ce89fb7a None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:04.535653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87030dc6-503c-44a9-9f6a-fcb5ce89fb7a None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:04.535920 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87030dc6-503c-44a9-9f6a-fcb5ce89fb7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:04.544755 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87030dc6-503c-44a9-9f6a-fcb5ce89fb7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:04.546527 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87030dc6-503c-44a9-9f6a-fcb5ce89fb7a None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:04.560131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-87030dc6-503c-44a9-9f6a-fcb5ce89fb7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:14:04.560131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:04.560131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:04.560131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:04.560131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:04.560131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:04.560131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:04.560131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:04.560131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:04.560131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:04.560131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:04.560131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:04.560131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:14:04.560131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:14:04.560131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:14:04.561225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:14:04.561225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:04.561225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:04.561225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:14:04.561225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:14:04.561225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:14:04.561225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:14:04.561225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:14:04.561225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:14:04.561225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:14:04.561225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:14:04.561225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:14:04.561225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:14:04.561225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:14:04.561225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:14:04.561225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:14:04.561225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:14:04.562325 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:14:04.562325 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:14:04.562325 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:14:04.562325 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:14:04.562325 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:14:04.562325 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:14:04.562325 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:14:04.562325 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:14:04.564242 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 489/973] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:14:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:14:04.569713 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87030dc6-503c-44a9-9f6a-fcb5ce89fb7a None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:04.589209 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:04.614922 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 485/974] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:14:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:14:04.675400 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87030dc6-503c-44a9-9f6a-fcb5ce89fb7a None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:04.713023 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabde603-09f8-468a-845c-9dc87be5fdd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:04.713635 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabde603-09f8-468a-845c-9dc87be5fdd6 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:04.713906 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabde603-09f8-468a-845c-9dc87be5fdd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:04.714170 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabde603-09f8-468a-845c-9dc87be5fdd6 None admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:04.714892 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabde603-09f8-468a-845c-9dc87be5fdd6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:04.715195 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabde603-09f8-468a-845c-9dc87be5fdd6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:04.716621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabde603-09f8-468a-845c-9dc87be5fdd6 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:04.727299 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:04.729434 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 490/975] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 14:14:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:14:04.735413 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a3fa7-33dd-4706-841e-d137a0f365e4 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:04.845627 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a3fa7-33dd-4706-841e-d137a0f365e4 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:04.885725 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4772ef-9786-406c-ab15-163864886d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:04.886643 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4772ef-9786-406c-ab15-163864886d26 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:04.887109 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4772ef-9786-406c-ab15-163864886d26 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:04.887523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4772ef-9786-406c-ab15-163864886d26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:04.888427 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4772ef-9786-406c-ab15-163864886d26 None admin] RBAC: Adding query filter params (role.id=30791f3f2a7841d2a80aab92cf4f23f5, scope.project.id=36a3ae7bab25471fb989136150b6438c, user.id=82328b7fd50649a485c6eaf27c5597c9) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:04.888899 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4772ef-9786-406c-ab15-163864886d26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:04.890549 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4772ef-9786-406c-ab15-163864886d26 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:04.897725 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 486/976] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:14:04 2022] GET /identity/v3/role_assignments?user.id=82328b7fd50649a485c6eaf27c5597c9&scope.project.id=36a3ae7bab25471fb989136150b6438c&role.id=30791f3f2a7841d2a80aab92cf4f23f5 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:04.913714 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabde603-09f8-468a-845c-9dc87be5fdd6 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:05.025956 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabde603-09f8-468a-845c-9dc87be5fdd6 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:05.068289 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab5dff-db47-4660-9b2a-817d5e77daa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:05.068906 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab5dff-db47-4660-9b2a-817d5e77daa8 None admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:05.069178 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab5dff-db47-4660-9b2a-817d5e77daa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:05.069443 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab5dff-db47-4660-9b2a-817d5e77daa8 None admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:05.070085 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ab5dff-db47-4660-9b2a-817d5e77daa8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:05.071588 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ab5dff-db47-4660-9b2a-817d5e77daa8 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:05.074130 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: WARNING keystone.server.flask.common [None req-e8ab5dff-db47-4660-9b2a-817d5e77daa8 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 24 14:14:05.089077 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.sql.core [None req-e8ab5dff-db47-4660-9b2a-817d5e77daa8 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 24 14:14:05.089077 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 24 14:14:05.089077 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [parameters: {'user_id': 'e19782677ddf4a73a410456761079a4b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 24 14:14:05.089077 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81496) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 24 14:14:05.096078 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-e8ab5dff-db47-4660-9b2a-817d5e77daa8 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 24 14:14:05.096078 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:05.096078 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 24 14:14:05.096078 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 24 14:14:05.096078 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 24 14:14:05.096078 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 24 14:14:05.096078 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 24 14:14:05.096078 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application result = self._query(query) Sep 24 14:14:05.096078 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 24 14:14:05.096078 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application conn.query(q) Sep 24 14:14:05.096078 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 24 14:14:05.096078 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 24 14:14:05.096078 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 24 14:14:05.096078 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application result.read() Sep 24 14:14:05.096078 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 24 14:14:05.098125 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 24 14:14:05.098125 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 24 14:14:05.098125 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 24 14:14:05.098125 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 24 14:14:05.098125 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 24 14:14:05.098125 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 24 14:14:05.098125 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 24 14:14:05.098125 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 24 14:14:05.098125 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:14:05.098125 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 24 14:14:05.098125 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:14:05.098125 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:05.098125 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 24 14:14:05.098125 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 24 14:14:05.098125 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 24 14:14:05.098125 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 24 14:14:05.098125 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 24 14:14:05.098125 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application next(self.gen) Sep 24 14:14:05.100169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 24 14:14:05.100169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application yield resource Sep 24 14:14:05.100169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 24 14:14:05.100169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application next(self.gen) Sep 24 14:14:05.100169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 24 14:14:05.100169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self.session.rollback() Sep 24 14:14:05.100169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 24 14:14:05.100169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self.force_reraise() Sep 24 14:14:05.100169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 24 14:14:05.100169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise self.value Sep 24 14:14:05.100169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 24 14:14:05.100169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 24 14:14:05.100169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 24 14:14:05.100169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application session.commit() Sep 24 14:14:05.100169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 24 14:14:05.100169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 24 14:14:05.100169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 24 14:14:05.103421 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self._prepare_impl() Sep 24 14:14:05.103421 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 24 14:14:05.103421 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self.session.flush() Sep 24 14:14:05.103421 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 24 14:14:05.103421 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self._flush(objects) Sep 24 14:14:05.103421 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 24 14:14:05.103421 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 24 14:14:05.103421 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 24 14:14:05.103421 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application compat.raise_( Sep 24 14:14:05.103421 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 24 14:14:05.103421 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception Sep 24 14:14:05.103421 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 24 14:14:05.103421 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application flush_context.execute() Sep 24 14:14:05.103421 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 24 14:14:05.103421 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rec.execute(self) Sep 24 14:14:05.103421 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 24 14:14:05.103421 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 24 14:14:05.108253 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 24 14:14:05.108253 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 24 14:14:05.108253 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 24 14:14:05.108253 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 24 14:14:05.108253 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 24 14:14:05.108253 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 24 14:14:05.108253 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 24 14:14:05.108253 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 24 14:14:05.108253 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 24 14:14:05.108253 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 24 14:14:05.108253 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 24 14:14:05.108253 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 24 14:14:05.108253 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 24 14:14:05.108253 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 24 14:14:05.108253 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 24 14:14:05.108253 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception Sep 24 14:14:05.111264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 24 14:14:05.111264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 24 14:14:05.111264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 24 14:14:05.111264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 24 14:14:05.111264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 24 14:14:05.111264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application result = self._query(query) Sep 24 14:14:05.111264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 24 14:14:05.111264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application conn.query(q) Sep 24 14:14:05.111264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 24 14:14:05.111264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 24 14:14:05.111264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 24 14:14:05.111264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application result.read() Sep 24 14:14:05.111264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 24 14:14:05.111264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 24 14:14:05.111264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 24 14:14:05.111264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 24 14:14:05.113236 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 24 14:14:05.113236 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 24 14:14:05.113236 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 24 14:14:05.113236 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 24 14:14:05.113236 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 24 14:14:05.113236 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 24 14:14:05.113236 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [parameters: {'user_id': 'e19782677ddf4a73a410456761079a4b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 24 14:14:05.113236 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 24 14:14:05.113236 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:14:05.113236 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:14:05.113236 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:14:05.113236 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:05.113236 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:05.113236 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:05.113236 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:05.115189 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:05.115189 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:05.115189 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:05.115189 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:05.115189 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:05.115189 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:05.115189 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:05.115189 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 24 14:14:05.115189 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 24 14:14:05.115189 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:05.115189 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:05.115189 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:14:05.115189 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:14:05.115189 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:14:05.115189 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:14:05.115189 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 24 14:14:05.115189 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 24 14:14:05.117775 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 24 14:14:05.117775 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 24 14:14:05.117775 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 24 14:14:05.117775 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 24 14:14:05.117775 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 24 14:14:05.117775 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:14:05.117775 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 491/977] 10.222.0.52 () {60 vars in 1131 bytes} [Sat Sep 24 14:14:04 2022] POST /identity/v3/users => generated 160 bytes in 184 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:14:05.119486 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4772ef-9786-406c-ab15-163864886d26 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:05.187157 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:05.238675 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4772ef-9786-406c-ab15-163864886d26 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:05.275230 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8235be0f-6a9e-4689-b1bd-13ca3927097e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:05.275865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235be0f-6a9e-4689-b1bd-13ca3927097e None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:05.276144 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235be0f-6a9e-4689-b1bd-13ca3927097e None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:05.276465 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235be0f-6a9e-4689-b1bd-13ca3927097e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:05.284452 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8235be0f-6a9e-4689-b1bd-13ca3927097e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:05.285887 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8235be0f-6a9e-4689-b1bd-13ca3927097e None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:05.297470 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-8235be0f-6a9e-4689-b1bd-13ca3927097e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 24 14:14:05.297470 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:05.297470 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:05.297470 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:05.297470 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:05.297470 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:05.297470 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:05.297470 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:05.297470 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:05.297470 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:05.297470 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:05.297470 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:05.297470 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:14:05.297470 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:14:05.297470 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:14:05.297470 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:14:05.298430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:05.298430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:05.298430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:14:05.298430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:14:05.298430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:14:05.298430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:14:05.298430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:14:05.298430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:14:05.298430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:14:05.298430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:14:05.298430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:14:05.298430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:14:05.298430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:14:05.298430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:14:05.298430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:14:05.298430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:14:05.298430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:14:05.298430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:14:05.299382 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:14:05.299382 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:14:05.299382 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:14:05.299382 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:14:05.299382 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:14:05.299382 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:14:05.299382 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:14:05.299382 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:14:05.299382 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 24 14:14:05.299382 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:05.299382 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 487/978] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:14:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:14:05.304710 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab5dff-db47-4660-9b2a-817d5e77daa8 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:05.362861 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:05.412548 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab5dff-db47-4660-9b2a-817d5e77daa8 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:05.447596 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f686cf3f-737c-4e1d-986a-9727c58bbece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:05.448478 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686cf3f-737c-4e1d-986a-9727c58bbece None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:05.448898 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686cf3f-737c-4e1d-986a-9727c58bbece None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:05.449310 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686cf3f-737c-4e1d-986a-9727c58bbece None admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:05.450201 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f686cf3f-737c-4e1d-986a-9727c58bbece None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:05.450646 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f686cf3f-737c-4e1d-986a-9727c58bbece None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:05.452440 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f686cf3f-737c-4e1d-986a-9727c58bbece None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:05.490582 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 492/979] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 14:14:05 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:06.284305 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a1e59-235d-4494-89ce-32df9ad49713 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:06.284824 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a1e59-235d-4494-89ce-32df9ad49713 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:06.284933 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a1e59-235d-4494-89ce-32df9ad49713 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:06.286276 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 488/980] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:14:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:14:06.292397 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a91297-c280-4131-b493-196d666b6e98 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:06.292791 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a91297-c280-4131-b493-196d666b6e98 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:06.292953 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a91297-c280-4131-b493-196d666b6e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:06.311832 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-f8a91297-c280-4131-b493-196d666b6e98 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:14:06.313338 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-f8a91297-c280-4131-b493-196d666b6e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:14:06.384170 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 493/981] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:14:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:14:06.455679 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea04827-ff17-4e86-8cd2-9d1abf66beb7 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:06.456296 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea04827-ff17-4e86-8cd2-9d1abf66beb7 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:06.456766 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea04827-ff17-4e86-8cd2-9d1abf66beb7 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:06.458540 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 489/982] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:14:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:14:06.463623 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c642e17-d553-4f32-a04a-49f626428e33 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:06.463916 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c642e17-d553-4f32-a04a-49f626428e33 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:06.464192 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c642e17-d553-4f32-a04a-49f626428e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:06.483448 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-7c642e17-d553-4f32-a04a-49f626428e33 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:14:06.484623 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-7c642e17-d553-4f32-a04a-49f626428e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:14:06.527995 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:06.557989 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 494/983] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:14:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:14:06.653721 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea04827-ff17-4e86-8cd2-9d1abf66beb7 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:06.756057 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea04827-ff17-4e86-8cd2-9d1abf66beb7 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:06.764955 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:06.791299 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4689e473-5dd8-4d85-95e9-7123bf88ae37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:06.791912 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689e473-5dd8-4d85-95e9-7123bf88ae37 None admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:06.792196 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689e473-5dd8-4d85-95e9-7123bf88ae37 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:06.792459 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689e473-5dd8-4d85-95e9-7123bf88ae37 None admin] PATH_INFO: `/v3/services` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:06.793107 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4689e473-5dd8-4d85-95e9-7123bf88ae37 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:06.794465 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4689e473-5dd8-4d85-95e9-7123bf88ae37 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:06.810944 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 490/984] 10.222.0.52 () {60 vars in 1141 bytes} [Sat Sep 24 14:14:06 2022] POST /identity/v3/services => generated 242 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:14:06.827255 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c642e17-d553-4f32-a04a-49f626428e33 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:06.939437 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c642e17-d553-4f32-a04a-49f626428e33 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:06.992876 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2860578-b9f3-4d83-af1f-325e2009c583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:06.993518 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2860578-b9f3-4d83-af1f-325e2009c583 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:06.993795 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2860578-b9f3-4d83-af1f-325e2009c583 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:06.994061 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2860578-b9f3-4d83-af1f-325e2009c583 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:06.994639 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2860578-b9f3-4d83-af1f-325e2009c583 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:06.996041 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2860578-b9f3-4d83-af1f-325e2009c583 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:07.004942 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-c2860578-b9f3-4d83-af1f-325e2009c583 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 14:14:07.004942 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:07.004942 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:14:07.004942 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:14:07.004942 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:14:07.004942 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:14:07.004942 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:14:07.004942 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:14:07.004942 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 14:14:07.004942 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:14:07.004942 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:14:07.004942 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:14:07.004942 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:07.004942 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:07.004942 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:07.004942 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:07.004942 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:07.005992 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:07.005992 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:07.005992 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:07.005992 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:07.005992 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:07.005992 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:07.005992 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:14:07.005992 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:14:07.005992 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:14:07.005992 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:14:07.005992 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:07.005992 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:07.005992 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:14:07.005992 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:14:07.005992 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:14:07.005992 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:14:07.006838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:14:07.006838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:14:07.006838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:14:07.006838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:14:07.006838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:14:07.006838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:14:07.006838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:14:07.006838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:14:07.006838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:14:07.006838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:14:07.006838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:14:07.006838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:14:07.006838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 14:14:07.006838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:14:07.010883 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 495/985] 10.222.0.52 () {56 vars in 1126 bytes} [Sat Sep 24 14:14:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:14:07.012903 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4689e473-5dd8-4d85-95e9-7123bf88ae37 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:07.114709 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4689e473-5dd8-4d85-95e9-7123bf88ae37 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:07.152044 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b4c639-64a5-4dfc-a9d0-c225f7824b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:07.152671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4c639-64a5-4dfc-a9d0-c225f7824b2b None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:07.152993 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4c639-64a5-4dfc-a9d0-c225f7824b2b None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:07.153454 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4c639-64a5-4dfc-a9d0-c225f7824b2b None admin] PATH_INFO: `/v3/services` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:07.154567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b4c639-64a5-4dfc-a9d0-c225f7824b2b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:07.155147 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b4c639-64a5-4dfc-a9d0-c225f7824b2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:07.156910 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b4c639-64a5-4dfc-a9d0-c225f7824b2b None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:07.163159 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 491/986] 10.222.0.52 () {56 vars in 1122 bytes} [Sat Sep 24 14:14:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:14:07.169620 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2860578-b9f3-4d83-af1f-325e2009c583 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:07.204726 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:07.252055 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556756f1-8cdf-4b4e-82ae-72ea0a4d9c77 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:07.252370 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556756f1-8cdf-4b4e-82ae-72ea0a4d9c77 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:07.252626 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556756f1-8cdf-4b4e-82ae-72ea0a4d9c77 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:07.254096 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 492/987] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:14:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:14:07.259449 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479a60c-7ae0-4b32-849d-8c13cd76c0e3 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:07.259971 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479a60c-7ae0-4b32-849d-8c13cd76c0e3 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:07.260390 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479a60c-7ae0-4b32-849d-8c13cd76c0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:07.278557 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2860578-b9f3-4d83-af1f-325e2009c583 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:07.280534 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-1479a60c-7ae0-4b32-849d-8c13cd76c0e3 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:14:07.281738 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-1479a60c-7ae0-4b32-849d-8c13cd76c0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:14:07.311826 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:07.315606 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5716c3e9-1972-49f9-95d3-e5a481dc3ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:07.316215 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716c3e9-1972-49f9-95d3-e5a481dc3ea8 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:07.316500 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716c3e9-1972-49f9-95d3-e5a481dc3ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:07.316766 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716c3e9-1972-49f9-95d3-e5a481dc3ea8 None admin] PATH_INFO: `/v3/services` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:07.317395 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5716c3e9-1972-49f9-95d3-e5a481dc3ea8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:07.317697 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5716c3e9-1972-49f9-95d3-e5a481dc3ea8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:07.319403 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5716c3e9-1972-49f9-95d3-e5a481dc3ea8 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:07.325061 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 496/988] 10.222.0.52 () {56 vars in 1122 bytes} [Sat Sep 24 14:14:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:14:07.331191 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5716c3e9-1972-49f9-95d3-e5a481dc3ea8 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:07.369908 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 493/989] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:14:07 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:14:07.437608 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:07.441608 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5716c3e9-1972-49f9-95d3-e5a481dc3ea8 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:07.475634 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062ae6e8-7d3b-4426-9a6c-68107f56a121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:07.476235 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ae6e8-7d3b-4426-9a6c-68107f56a121 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:07.476509 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ae6e8-7d3b-4426-9a6c-68107f56a121 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:07.476777 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ae6e8-7d3b-4426-9a6c-68107f56a121 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:07.478015 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062ae6e8-7d3b-4426-9a6c-68107f56a121 None admin] RBAC: Adding query filter params (interface=admin, service_id=678c93a260fa4a66b1cf09f60228b78f, region_id=RegionOne) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:07.478336 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062ae6e8-7d3b-4426-9a6c-68107f56a121 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:07.479716 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062ae6e8-7d3b-4426-9a6c-68107f56a121 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:07.484837 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 497/990] 10.222.0.52 () {56 vars in 1249 bytes} [Sat Sep 24 14:14:07 2022] GET /identity/v3/endpoints?service_id=678c93a260fa4a66b1cf09f60228b78f&interface=admin®ion_id=RegionOne => generated 193 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:07.490243 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1479a60c-7ae0-4b32-849d-8c13cd76c0e3 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:07.594238 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1479a60c-7ae0-4b32-849d-8c13cd76c0e3 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:07.628606 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ebe2f1-0258-4922-92dc-f743c2d4e55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:07.629134 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ebe2f1-0258-4922-92dc-f743c2d4e55e None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:07.629407 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ebe2f1-0258-4922-92dc-f743c2d4e55e None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:07.629673 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ebe2f1-0258-4922-92dc-f743c2d4e55e None admin] PATH_INFO: `/v3/services` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:07.630289 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ebe2f1-0258-4922-92dc-f743c2d4e55e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:07.631682 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ebe2f1-0258-4922-92dc-f743c2d4e55e None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:07.643109 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 494/991] 10.222.0.52 () {56 vars in 1087 bytes} [Sat Sep 24 14:14:07 2022] GET /identity/v3/services => generated 1727 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:14:07.674524 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062ae6e8-7d3b-4426-9a6c-68107f56a121 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:07.788147 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062ae6e8-7d3b-4426-9a6c-68107f56a121 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:07.824644 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d97b47-586f-46e4-bceb-668eae2eafd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:07.825283 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d97b47-586f-46e4-bceb-668eae2eafd0 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:07.825562 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d97b47-586f-46e4-bceb-668eae2eafd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:07.825835 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d97b47-586f-46e4-bceb-668eae2eafd0 None admin] PATH_INFO: `/v3/services/image` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:07.826420 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d97b47-586f-46e4-bceb-668eae2eafd0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:07.827845 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d97b47-586f-46e4-bceb-668eae2eafd0 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:07.837347 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-28d97b47-586f-46e4-bceb-668eae2eafd0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 14:14:07.837347 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:07.837347 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:14:07.837347 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:14:07.837347 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:14:07.837347 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:14:07.837347 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:14:07.837347 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:14:07.837347 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 14:14:07.837347 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:14:07.837347 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:14:07.837347 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:14:07.837347 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:07.837347 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:07.837347 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:07.837347 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:07.837347 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:07.838315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:07.838315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:07.838315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:07.838315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:07.838315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:07.838315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:07.838315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:14:07.838315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:14:07.838315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:14:07.838315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:14:07.838315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:07.838315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:07.838315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:14:07.838315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:14:07.838315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:14:07.838315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:14:07.839505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:14:07.839505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:14:07.839505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:14:07.839505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:14:07.839505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:14:07.839505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:14:07.839505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:14:07.839505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:14:07.839505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:14:07.839505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:14:07.839505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:14:07.839505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:14:07.839505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 14:14:07.839505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:14:07.839505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 498/992] 10.222.0.52 () {56 vars in 1105 bytes} [Sat Sep 24 14:14:07 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:14:07.844366 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ebe2f1-0258-4922-92dc-f743c2d4e55e None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:07.873545 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:07.948818 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ebe2f1-0258-4922-92dc-f743c2d4e55e None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:07.982178 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e725c90d-4b6f-4c6a-a610-355b5663d6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:07.982780 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725c90d-4b6f-4c6a-a610-355b5663d6fe None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:07.983080 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725c90d-4b6f-4c6a-a610-355b5663d6fe None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:07.983347 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725c90d-4b6f-4c6a-a610-355b5663d6fe None admin] PATH_INFO: `/v3/services` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:07.983970 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e725c90d-4b6f-4c6a-a610-355b5663d6fe None admin] RBAC: Adding query filter params (name=image) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:07.984266 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e725c90d-4b6f-4c6a-a610-355b5663d6fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:07.985616 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e725c90d-4b6f-4c6a-a610-355b5663d6fe None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:07.994893 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 495/993] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:14:07 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:07.996057 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d97b47-586f-46e4-bceb-668eae2eafd0 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:08.024849 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:08.098875 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d97b47-586f-46e4-bceb-668eae2eafd0 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:08.133293 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd6d68a-bfdb-468c-9920-2226da43379e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:08.134513 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6d68a-bfdb-468c-9920-2226da43379e None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:08.134793 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6d68a-bfdb-468c-9920-2226da43379e None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:08.135086 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6d68a-bfdb-468c-9920-2226da43379e None admin] PATH_INFO: `/v3/services` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:08.135710 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd6d68a-bfdb-468c-9920-2226da43379e None admin] RBAC: Adding query filter params (type=image) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:08.136006 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd6d68a-bfdb-468c-9920-2226da43379e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:08.137372 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd6d68a-bfdb-468c-9920-2226da43379e None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:08.146905 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 499/994] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:14:07 2022] GET /identity/v3/services?type=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:14:09.042585 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0a684-ea33-417a-a5b9-8f2434fe6d0d None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:09.043176 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0a684-ea33-417a-a5b9-8f2434fe6d0d None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:09.043176 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0a684-ea33-417a-a5b9-8f2434fe6d0d None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:09.044546 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 496/995] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:14:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:14:09.049610 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d72309-b946-4ca8-a50a-c856f812c6f1 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:09.049981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d72309-b946-4ca8-a50a-c856f812c6f1 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:09.050135 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d72309-b946-4ca8-a50a-c856f812c6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:09.068564 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-35d72309-b946-4ca8-a50a-c856f812c6f1 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:14:09.069735 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-35d72309-b946-4ca8-a50a-c856f812c6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:14:09.135918 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 500/996] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:14:09 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:14:09.402285 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0a684-ea33-417a-a5b9-8f2434fe6d0d None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:09.497051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:09.508173 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0a684-ea33-417a-a5b9-8f2434fe6d0d None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:09.542366 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab37409-8320-4762-bc92-d1d1d0c5a9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:09.543023 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab37409-8320-4762-bc92-d1d1d0c5a9bf None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:09.543302 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab37409-8320-4762-bc92-d1d1d0c5a9bf None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:09.543573 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab37409-8320-4762-bc92-d1d1d0c5a9bf None admin] PATH_INFO: `/v3/services/compute` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:09.544148 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab37409-8320-4762-bc92-d1d1d0c5a9bf None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:09.545505 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab37409-8320-4762-bc92-d1d1d0c5a9bf None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:09.554241 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-dab37409-8320-4762-bc92-d1d1d0c5a9bf None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 14:14:09.554241 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:09.554241 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:14:09.554241 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:14:09.554241 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:14:09.554241 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:14:09.554241 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:14:09.554241 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:14:09.554241 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 14:14:09.554241 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:09.554241 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:14:09.554241 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:09.554241 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:09.554241 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:09.554241 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:09.554241 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:09.554241 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:09.555720 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:09.555720 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:09.555720 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:09.555720 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:09.555720 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:09.555720 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:09.555720 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:14:09.555720 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:14:09.555720 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:14:09.555720 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:14:09.555720 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:09.555720 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:09.555720 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:14:09.555720 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:14:09.555720 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:14:09.555720 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:14:09.557389 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:14:09.557389 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:14:09.557389 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:14:09.557389 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:14:09.557389 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:14:09.557389 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:14:09.557389 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:14:09.557389 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:14:09.557389 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:14:09.557389 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:14:09.557389 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:14:09.557389 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:14:09.557389 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 14:14:09.557389 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:09.557389 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 497/997] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:14:09 2022] GET /identity/v3/services/compute => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:14:09.561800 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d72309-b946-4ca8-a50a-c856f812c6f1 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:09.583473 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:09.663121 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d72309-b946-4ca8-a50a-c856f812c6f1 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:09.697015 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5665df9d-fb22-4f8d-a0ed-69f4ccdcd54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:09.697637 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665df9d-fb22-4f8d-a0ed-69f4ccdcd54e None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:09.697914 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665df9d-fb22-4f8d-a0ed-69f4ccdcd54e None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:09.698182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665df9d-fb22-4f8d-a0ed-69f4ccdcd54e None admin] PATH_INFO: `/v3/services` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:09.698857 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5665df9d-fb22-4f8d-a0ed-69f4ccdcd54e None admin] RBAC: Adding query filter params (name=compute) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:09.699148 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5665df9d-fb22-4f8d-a0ed-69f4ccdcd54e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:09.700563 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5665df9d-fb22-4f8d-a0ed-69f4ccdcd54e None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:09.710909 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 501/998] 10.222.0.52 () {56 vars in 1112 bytes} [Sat Sep 24 14:14:09 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:14:09.712229 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab37409-8320-4762-bc92-d1d1d0c5a9bf None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:09.818129 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab37409-8320-4762-bc92-d1d1d0c5a9bf None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:09.824952 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61bd51-4298-4c9f-a8fd-12c5d3e96ad4 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:09.825237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61bd51-4298-4c9f-a8fd-12c5d3e96ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:09.825508 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61bd51-4298-4c9f-a8fd-12c5d3e96ad4 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:09.827003 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 502/999] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:14:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:14:09.831829 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94d364-79b3-4505-b4d8-e0ef18875ef3 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:09.832108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94d364-79b3-4505-b4d8-e0ef18875ef3 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:09.832379 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94d364-79b3-4505-b4d8-e0ef18875ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:09.850178 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-af94d364-79b3-4505-b4d8-e0ef18875ef3 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:14:09.851375 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-af94d364-79b3-4505-b4d8-e0ef18875ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:14:09.857295 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98af676-cfa8-4ed1-889d-67744eae1cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:09.857898 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98af676-cfa8-4ed1-889d-67744eae1cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:09.858173 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98af676-cfa8-4ed1-889d-67744eae1cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:09.858442 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98af676-cfa8-4ed1-889d-67744eae1cf5 None admin] PATH_INFO: `/v3/services` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:09.859092 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98af676-cfa8-4ed1-889d-67744eae1cf5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:09.859388 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98af676-cfa8-4ed1-889d-67744eae1cf5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:09.860741 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98af676-cfa8-4ed1-889d-67744eae1cf5 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:09.865601 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 498/1000] 10.222.0.52 () {56 vars in 1112 bytes} [Sat Sep 24 14:14:09 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:09.871735 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98af676-cfa8-4ed1-889d-67744eae1cf5 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:09.891768 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:09.931243 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 503/1001] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:14:09 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:14:09.983381 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98af676-cfa8-4ed1-889d-67744eae1cf5 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:10.017561 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82be240-9500-4bba-97a2-45beaa8dc070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:10.018166 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82be240-9500-4bba-97a2-45beaa8dc070 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:10.018443 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82be240-9500-4bba-97a2-45beaa8dc070 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:10.018713 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82be240-9500-4bba-97a2-45beaa8dc070 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:10.019435 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82be240-9500-4bba-97a2-45beaa8dc070 None admin] RBAC: Adding query filter params (interface=public, service_id=9d6d470932e34517a3d42b1be1461b5b, region_id=RegionOne) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:10.019741 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82be240-9500-4bba-97a2-45beaa8dc070 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:10.021122 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82be240-9500-4bba-97a2-45beaa8dc070 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:10.026758 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 499/1002] 10.222.0.52 () {56 vars in 1251 bytes} [Sat Sep 24 14:14:09 2022] GET /identity/v3/endpoints?service_id=9d6d470932e34517a3d42b1be1461b5b&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:14:10.032534 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94d364-79b3-4505-b4d8-e0ef18875ef3 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:10.049073 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:10.136181 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94d364-79b3-4505-b4d8-e0ef18875ef3 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:10.169929 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679feafa-2798-467f-9826-5aa70e0dfbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:10.170557 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679feafa-2798-467f-9826-5aa70e0dfbdb None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:10.170840 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679feafa-2798-467f-9826-5aa70e0dfbdb None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:10.171116 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679feafa-2798-467f-9826-5aa70e0dfbdb None admin] PATH_INFO: `/v3/services` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:10.171725 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679feafa-2798-467f-9826-5aa70e0dfbdb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:10.173093 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679feafa-2798-467f-9826-5aa70e0dfbdb None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:10.187157 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 504/1003] 10.222.0.52 () {56 vars in 1087 bytes} [Sat Sep 24 14:14:10 2022] GET /identity/v3/services => generated 1727 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:14:10.196831 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82be240-9500-4bba-97a2-45beaa8dc070 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:10.292116 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:10.312945 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82be240-9500-4bba-97a2-45beaa8dc070 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:10.348687 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bbf6a5-22d3-45c6-8e42-9d278ffe27ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:10.349328 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bbf6a5-22d3-45c6-8e42-9d278ffe27ad None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:10.349602 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bbf6a5-22d3-45c6-8e42-9d278ffe27ad None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:10.349866 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bbf6a5-22d3-45c6-8e42-9d278ffe27ad None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:10.350456 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bbf6a5-22d3-45c6-8e42-9d278ffe27ad None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:10.351936 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bbf6a5-22d3-45c6-8e42-9d278ffe27ad None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:10.353560 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0cd8a1-11b7-498f-9126-a8df7e9700ae None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:10.353852 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0cd8a1-11b7-498f-9126-a8df7e9700ae None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:10.354126 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0cd8a1-11b7-498f-9126-a8df7e9700ae None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:10.355545 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 505/1004] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:14:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:14:10.360835 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f79586-5af2-448a-a6d9-5f4699127a80 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:10.361105 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f79586-5af2-448a-a6d9-5f4699127a80 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:10.361381 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f79586-5af2-448a-a6d9-5f4699127a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:10.362591 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-68bbf6a5-22d3-45c6-8e42-9d278ffe27ad None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 14:14:10.362591 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:10.362591 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:14:10.362591 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:14:10.362591 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:14:10.362591 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:14:10.362591 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:14:10.362591 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:14:10.362591 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 14:14:10.362591 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:10.362591 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:14:10.362591 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:10.362591 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:10.362591 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:10.362591 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:10.362591 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:10.362591 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:10.363702 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:10.363702 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:10.363702 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:10.363702 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:10.363702 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:10.363702 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:10.363702 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:14:10.363702 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:14:10.363702 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:14:10.363702 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:14:10.363702 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:10.363702 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:10.363702 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:14:10.363702 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:14:10.363702 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:14:10.363702 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:14:10.364538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:14:10.364538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:14:10.364538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:14:10.364538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:14:10.364538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:14:10.364538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:14:10.364538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:14:10.364538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:14:10.364538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:14:10.364538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:14:10.364538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:14:10.364538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:14:10.364538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 14:14:10.364538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:10.364538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 500/1005] 10.222.0.52 () {56 vars in 1126 bytes} [Sat Sep 24 14:14:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:14:10.369860 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bbf6a5-22d3-45c6-8e42-9d278ffe27ad None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:10.382426 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-d6f79586-5af2-448a-a6d9-5f4699127a80 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:14:10.383746 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-d6f79586-5af2-448a-a6d9-5f4699127a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:14:10.461239 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 506/1006] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:14:10 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:14:10.482845 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bbf6a5-22d3-45c6-8e42-9d278ffe27ad None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:10.518240 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87074e64-1d10-4922-9fdf-9f44bf99c9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:10.518882 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87074e64-1d10-4922-9fdf-9f44bf99c9d6 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:10.519217 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87074e64-1d10-4922-9fdf-9f44bf99c9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:10.519429 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87074e64-1d10-4922-9fdf-9f44bf99c9d6 None admin] PATH_INFO: `/v3/services` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:10.520179 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87074e64-1d10-4922-9fdf-9f44bf99c9d6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:10.520476 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87074e64-1d10-4922-9fdf-9f44bf99c9d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:10.521860 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87074e64-1d10-4922-9fdf-9f44bf99c9d6 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:10.527072 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 501/1007] 10.222.0.52 () {56 vars in 1122 bytes} [Sat Sep 24 14:14:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:14:10.532735 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f79586-5af2-448a-a6d9-5f4699127a80 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:10.543707 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:10.639858 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f79586-5af2-448a-a6d9-5f4699127a80 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:10.674275 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7790e2-56a5-4ac8-9481-226de5e6e259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:10.674907 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7790e2-56a5-4ac8-9481-226de5e6e259 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:10.675188 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7790e2-56a5-4ac8-9481-226de5e6e259 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:10.675459 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7790e2-56a5-4ac8-9481-226de5e6e259 None admin] PATH_INFO: `/v3/services` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:10.676089 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7790e2-56a5-4ac8-9481-226de5e6e259 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:10.676388 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7790e2-56a5-4ac8-9481-226de5e6e259 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:10.677750 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7790e2-56a5-4ac8-9481-226de5e6e259 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:10.688009 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 507/1008] 10.222.0.52 () {56 vars in 1122 bytes} [Sat Sep 24 14:14:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:10.689356 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87074e64-1d10-4922-9fdf-9f44bf99c9d6 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:10.702099 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:10.744943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7790e2-56a5-4ac8-9481-226de5e6e259 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:10.797802 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87074e64-1d10-4922-9fdf-9f44bf99c9d6 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:10.822756 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:10.842474 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b2eb4b-7da1-4864-b6be-92d5f4515ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:10.843115 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2eb4b-7da1-4864-b6be-92d5f4515ce7 None admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:10.843396 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2eb4b-7da1-4864-b6be-92d5f4515ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:10.843671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2eb4b-7da1-4864-b6be-92d5f4515ce7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:10.844677 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b2eb4b-7da1-4864-b6be-92d5f4515ce7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:10.846774 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b2eb4b-7da1-4864-b6be-92d5f4515ce7 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:10.854164 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7790e2-56a5-4ac8-9481-226de5e6e259 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:10.866223 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 502/1009] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:14:10 2022] POST /identity/v3/endpoints => generated 325 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:14:10.902722 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d55fca-a7e1-41cc-a981-99fa67403497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:10.903743 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d55fca-a7e1-41cc-a981-99fa67403497 None admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:10.904291 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d55fca-a7e1-41cc-a981-99fa67403497 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:10.904785 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d55fca-a7e1-41cc-a981-99fa67403497 None admin] PATH_INFO: `/v3/services` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:10.905759 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d55fca-a7e1-41cc-a981-99fa67403497 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:10.907531 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d55fca-a7e1-41cc-a981-99fa67403497 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:10.926931 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 508/1010] 10.222.0.52 () {60 vars in 1141 bytes} [Sat Sep 24 14:14:10 2022] POST /identity/v3/services => generated 242 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:14:12.381346 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703c1869-6289-4867-ba17-e8b19809a7f2 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:12.381862 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703c1869-6289-4867-ba17-e8b19809a7f2 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:12.381862 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703c1869-6289-4867-ba17-e8b19809a7f2 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:12.383431 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 503/1011] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:14:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:14:12.388445 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ef1a6-e5e2-48e4-9888-99f46497bcb6 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:12.388879 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ef1a6-e5e2-48e4-9888-99f46497bcb6 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:12.389026 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ef1a6-e5e2-48e4-9888-99f46497bcb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:12.408106 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-666ef1a6-e5e2-48e4-9888-99f46497bcb6 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:14:12.409537 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-666ef1a6-e5e2-48e4-9888-99f46497bcb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:14:12.482136 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 509/1012] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:14:12 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:14:12.752481 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703c1869-6289-4867-ba17-e8b19809a7f2 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:12.826175 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:12.861698 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703c1869-6289-4867-ba17-e8b19809a7f2 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:12.897185 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf80e653-ca20-4de7-a1bf-56ec19d7c89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:12.897810 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf80e653-ca20-4de7-a1bf-56ec19d7c89f None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:12.898077 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf80e653-ca20-4de7-a1bf-56ec19d7c89f None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:12.898344 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf80e653-ca20-4de7-a1bf-56ec19d7c89f None admin] PATH_INFO: `/v3/services/compute` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:12.898975 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf80e653-ca20-4de7-a1bf-56ec19d7c89f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:12.900319 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf80e653-ca20-4de7-a1bf-56ec19d7c89f None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:12.909561 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-cf80e653-ca20-4de7-a1bf-56ec19d7c89f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 14:14:12.909561 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:12.909561 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:14:12.909561 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:14:12.909561 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:14:12.909561 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:14:12.909561 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:14:12.909561 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:14:12.909561 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 14:14:12.909561 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:12.909561 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:14:12.909561 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:12.909561 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:12.909561 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:12.909561 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:12.909561 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:12.909561 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:12.910649 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:12.910649 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:12.910649 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:12.910649 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:12.910649 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:12.910649 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:12.910649 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:14:12.910649 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:14:12.910649 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:14:12.910649 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:14:12.910649 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:12.910649 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:12.910649 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:14:12.910649 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:14:12.910649 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:14:12.910649 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:14:12.916026 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:14:12.916026 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:14:12.916026 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:14:12.916026 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:14:12.916026 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:14:12.916026 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:14:12.916026 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:14:12.916026 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:14:12.916026 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:14:12.916026 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:14:12.916026 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:14:12.916026 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:14:12.916026 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 14:14:12.916026 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:12.916026 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 504/1013] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:14:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:14:12.918388 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666ef1a6-e5e2-48e4-9888-99f46497bcb6 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:12.976638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:13.027945 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666ef1a6-e5e2-48e4-9888-99f46497bcb6 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:13.036106 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4f134-a650-4fc5-ad18-1bf4b72a7b5e None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:13.036431 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4f134-a650-4fc5-ad18-1bf4b72a7b5e None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:13.036658 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4f134-a650-4fc5-ad18-1bf4b72a7b5e None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:13.038105 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 505/1014] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:14:13.044216 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43374f3-84bd-4ffc-9b5f-7b4974de0e5b None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:13.044502 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43374f3-84bd-4ffc-9b5f-7b4974de0e5b None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:13.044776 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43374f3-84bd-4ffc-9b5f-7b4974de0e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:13.063843 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-b43374f3-84bd-4ffc-9b5f-7b4974de0e5b None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:14:13.065021 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-b43374f3-84bd-4ffc-9b5f-7b4974de0e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:14:13.068196 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb0e21-948b-4f5e-a4a8-70b3c7465c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:13.068808 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb0e21-948b-4f5e-a4a8-70b3c7465c45 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:13.069082 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb0e21-948b-4f5e-a4a8-70b3c7465c45 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:13.069348 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb0e21-948b-4f5e-a4a8-70b3c7465c45 None admin] PATH_INFO: `/v3/services` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:13.069977 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fb0e21-948b-4f5e-a4a8-70b3c7465c45 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:13.070272 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fb0e21-948b-4f5e-a4a8-70b3c7465c45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:13.072193 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fb0e21-948b-4f5e-a4a8-70b3c7465c45 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:13.077018 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 510/1015] 10.222.0.52 () {56 vars in 1112 bytes} [Sat Sep 24 14:14:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:13.083255 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb0e21-948b-4f5e-a4a8-70b3c7465c45 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:13.142092 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 506/1016] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:14:13 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:14:13.148460 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f80e63-2df9-4601-90e6-616a772f9131 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:13.148966 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f80e63-2df9-4601-90e6-616a772f9131 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:13.149734 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f80e63-2df9-4601-90e6-616a772f9131 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:13.154851 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 507/1017] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:14:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:14:13.156646 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b798c-fef5-400f-a197-949c917594ea None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:13.157236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b798c-fef5-400f-a197-949c917594ea None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:13.157688 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b798c-fef5-400f-a197-949c917594ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:13.184734 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-ff0b798c-fef5-400f-a197-949c917594ea None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:14:13.186027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-ff0b798c-fef5-400f-a197-949c917594ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:14:13.192607 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb0e21-948b-4f5e-a4a8-70b3c7465c45 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:13.210519 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:13.227799 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb123076-4809-4148-a73b-0592d67abbd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:13.228408 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb123076-4809-4148-a73b-0592d67abbd6 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:13.228684 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb123076-4809-4148-a73b-0592d67abbd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:13.228954 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb123076-4809-4148-a73b-0592d67abbd6 None admin] PATH_INFO: `/v3/services` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:13.229579 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb123076-4809-4148-a73b-0592d67abbd6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:13.229879 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb123076-4809-4148-a73b-0592d67abbd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:13.231282 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb123076-4809-4148-a73b-0592d67abbd6 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:13.236445 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 511/1018] 10.222.0.52 () {56 vars in 1112 bytes} [Sat Sep 24 14:14:13 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:14:13.241913 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb123076-4809-4148-a73b-0592d67abbd6 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:13.251697 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:13.260079 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 508/1019] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:14:13 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:14:13.349218 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb123076-4809-4148-a73b-0592d67abbd6 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:13.383550 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3fc82-978b-4254-ae91-c347574a2041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:13.384152 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3fc82-978b-4254-ae91-c347574a2041 None admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:13.384429 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3fc82-978b-4254-ae91-c347574a2041 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:13.384697 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3fc82-978b-4254-ae91-c347574a2041 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:13.385349 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce3fc82-978b-4254-ae91-c347574a2041 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:13.386715 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce3fc82-978b-4254-ae91-c347574a2041 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:13.415281 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 512/1020] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:14:13 2022] POST /identity/v3/endpoints => generated 334 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:14:13.421901 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b798c-fef5-400f-a197-949c917594ea None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:13.528720 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3fc82-978b-4254-ae91-c347574a2041 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:13.533750 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b798c-fef5-400f-a197-949c917594ea None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:13.572374 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:13.578557 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13799019-1a47-48f4-b386-9b359dc4d097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:13.579241 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13799019-1a47-48f4-b386-9b359dc4d097 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:13.579524 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13799019-1a47-48f4-b386-9b359dc4d097 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:13.579794 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13799019-1a47-48f4-b386-9b359dc4d097 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:13.580926 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13799019-1a47-48f4-b386-9b359dc4d097 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:13.582377 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13799019-1a47-48f4-b386-9b359dc4d097 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:13.584193 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 509/1021] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:14:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:13.589795 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13799019-1a47-48f4-b386-9b359dc4d097 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:13.652275 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3fc82-978b-4254-ae91-c347574a2041 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:13.656960 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:13.692264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b35a0-3928-492e-a95a-691ddb36e888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:13.692470 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b35a0-3928-492e-a95a-691ddb36e888 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:13.692708 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b35a0-3928-492e-a95a-691ddb36e888 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:13.692984 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b35a0-3928-492e-a95a-691ddb36e888 None admin] PATH_INFO: `/v3/services/image` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:13.693563 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b35a0-3928-492e-a95a-691ddb36e888 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:13.698095 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b35a0-3928-492e-a95a-691ddb36e888 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:13.707522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-7a2b35a0-3928-492e-a95a-691ddb36e888 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 14:14:13.707522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:13.707522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:14:13.707522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:14:13.707522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:14:13.707522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:14:13.707522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:14:13.707522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:14:13.707522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 14:14:13.707522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:14:13.707522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:14:13.707522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:14:13.707522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:13.707522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:13.707522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:13.707522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:13.707522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:13.708534 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:13.708534 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:13.708534 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:13.708534 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:13.708534 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:13.708534 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:13.708534 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:14:13.708534 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:14:13.708534 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:14:13.708534 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:14:13.708534 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:13.708534 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:13.708534 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:14:13.708534 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:14:13.708534 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:14:13.708534 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:14:13.709754 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:14:13.709754 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:14:13.709754 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:14:13.709754 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:14:13.709754 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:14:13.709754 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:14:13.709754 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:14:13.709754 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:14:13.709754 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:14:13.709754 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:14:13.709754 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:14:13.709754 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:14:13.709754 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 14:14:13.709754 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:14:13.710508 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13799019-1a47-48f4-b386-9b359dc4d097 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:13.710593 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 513/1022] 10.222.0.52 () {56 vars in 1105 bytes} [Sat Sep 24 14:14:13 2022] GET /identity/v3/services/image => generated 86 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:14:13.716593 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b35a0-3928-492e-a95a-691ddb36e888 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:13.750938 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe447eb3-aa64-4167-a51f-3dee3d2d3a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:13.751716 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe447eb3-aa64-4167-a51f-3dee3d2d3a22 None admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:13.752200 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe447eb3-aa64-4167-a51f-3dee3d2d3a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:13.752791 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe447eb3-aa64-4167-a51f-3dee3d2d3a22 None admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:13.753457 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe447eb3-aa64-4167-a51f-3dee3d2d3a22 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:13.755599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe447eb3-aa64-4167-a51f-3dee3d2d3a22 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:13.790556 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 510/1023] 10.222.0.52 () {60 vars in 1141 bytes} [Sat Sep 24 14:14:13 2022] POST /identity/v3/projects => generated 311 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:14:13.844845 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b35a0-3928-492e-a95a-691ddb36e888 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:13.846932 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:13.887435 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc26ebac-557a-4b35-ad52-3a362f6375fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:13.888034 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26ebac-557a-4b35-ad52-3a362f6375fd None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:13.888315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26ebac-557a-4b35-ad52-3a362f6375fd None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:13.888591 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26ebac-557a-4b35-ad52-3a362f6375fd None admin] PATH_INFO: `/v3/services` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:13.889237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc26ebac-557a-4b35-ad52-3a362f6375fd None admin] RBAC: Adding query filter params (name=image) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:13.889540 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc26ebac-557a-4b35-ad52-3a362f6375fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:13.890975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc26ebac-557a-4b35-ad52-3a362f6375fd None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:13.896900 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 514/1024] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:14:13 2022] GET /identity/v3/services?name=image => generated 122 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:13.902600 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe447eb3-aa64-4167-a51f-3dee3d2d3a22 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:14.012105 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe447eb3-aa64-4167-a51f-3dee3d2d3a22 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:14.022968 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:14.049586 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a422c60-dbd8-4c29-b7d9-16690d0ee375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:14.050195 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a422c60-dbd8-4c29-b7d9-16690d0ee375 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:14.050474 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a422c60-dbd8-4c29-b7d9-16690d0ee375 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:14.050746 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a422c60-dbd8-4c29-b7d9-16690d0ee375 None admin] PATH_INFO: `/v3/services` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:14.051433 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a422c60-dbd8-4c29-b7d9-16690d0ee375 None admin] RBAC: Adding query filter params (type=image) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:14.051730 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a422c60-dbd8-4c29-b7d9-16690d0ee375 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:14.053086 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a422c60-dbd8-4c29-b7d9-16690d0ee375 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:14.058347 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 511/1025] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:14:13 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:14:14.064557 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc26ebac-557a-4b35-ad52-3a362f6375fd None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:14.175304 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc26ebac-557a-4b35-ad52-3a362f6375fd None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:14.210055 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e16152-8794-462a-8fc9-59fa656a573a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:14.210667 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e16152-8794-462a-8fc9-59fa656a573a None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:14.210975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e16152-8794-462a-8fc9-59fa656a573a None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:14.211247 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e16152-8794-462a-8fc9-59fa656a573a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:14.211981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e16152-8794-462a-8fc9-59fa656a573a None admin] RBAC: Adding query filter params (interface=public, service_id=5fd87ed4094d4b3e98c511aba762f933, region_id=RegionOne) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:14.212288 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e16152-8794-462a-8fc9-59fa656a573a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:14.213691 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e16152-8794-462a-8fc9-59fa656a573a None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:14.223546 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 515/1026] 10.222.0.52 () {56 vars in 1251 bytes} [Sat Sep 24 14:14:14 2022] GET /identity/v3/endpoints?service_id=5fd87ed4094d4b3e98c511aba762f933&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:14.225117 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a422c60-dbd8-4c29-b7d9-16690d0ee375 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:14.328277 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a422c60-dbd8-4c29-b7d9-16690d0ee375 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:14.361363 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7aa29b-b998-48cc-b7b9-6ad5f8e35cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:14.361973 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7aa29b-b998-48cc-b7b9-6ad5f8e35cb0 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:14.362376 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7aa29b-b998-48cc-b7b9-6ad5f8e35cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:14.362653 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7aa29b-b998-48cc-b7b9-6ad5f8e35cb0 None admin] PATH_INFO: `/v3/services` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:14.363305 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7aa29b-b998-48cc-b7b9-6ad5f8e35cb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:14.364661 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7aa29b-b998-48cc-b7b9-6ad5f8e35cb0 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:14.371798 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 512/1027] 10.222.0.52 () {56 vars in 1087 bytes} [Sat Sep 24 14:14:14 2022] GET /identity/v3/services => generated 1957 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:14:15.724776 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfcc8e-7578-47eb-9820-f419c722ad1c None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:15.725299 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfcc8e-7578-47eb-9820-f419c722ad1c None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:15.725375 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfcc8e-7578-47eb-9820-f419c722ad1c None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:15.726705 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 516/1028] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:14:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:14:15.731911 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba156b0f-78bd-49ab-83d9-4d4e0008ce6f None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:15.732293 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba156b0f-78bd-49ab-83d9-4d4e0008ce6f None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:15.732467 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba156b0f-78bd-49ab-83d9-4d4e0008ce6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:15.750934 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-ba156b0f-78bd-49ab-83d9-4d4e0008ce6f None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:14:15.752114 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-ba156b0f-78bd-49ab-83d9-4d4e0008ce6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:14:15.785725 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:15.834969 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 513/1029] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:14:15 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:14:16.009274 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b66aed-be06-4d8c-b561-e61b20377136 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:16.009448 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b66aed-be06-4d8c-b561-e61b20377136 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:16.009721 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b66aed-be06-4d8c-b561-e61b20377136 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:16.011131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 517/1030] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:14:16.016365 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb26bd-355f-4e9a-a125-e420bfaa4555 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:16.016654 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb26bd-355f-4e9a-a125-e420bfaa4555 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:16.016930 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb26bd-355f-4e9a-a125-e420bfaa4555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:16.035673 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-e2bb26bd-355f-4e9a-a125-e420bfaa4555 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:14:16.036846 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-e2bb26bd-355f-4e9a-a125-e420bfaa4555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:14:16.099371 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b66aed-be06-4d8c-b561-e61b20377136 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:16.104243 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 514/1031] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:14:16 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:14:16.129286 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:16.204127 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b66aed-be06-4d8c-b561-e61b20377136 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:16.238644 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f969ec13-c80c-4939-b148-1d2ebb5d71fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:16.239315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969ec13-c80c-4939-b148-1d2ebb5d71fc None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:16.239592 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969ec13-c80c-4939-b148-1d2ebb5d71fc None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:16.239925 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969ec13-c80c-4939-b148-1d2ebb5d71fc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:16.250799 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f969ec13-c80c-4939-b148-1d2ebb5d71fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:14:16.255621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-f969ec13-c80c-4939-b148-1d2ebb5d71fc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 14:14:16.255621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:16.255621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:16.255621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:16.255621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:16.255621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:16.255621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:16.255621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:16.255621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:16.255621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:16.255621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:16.255621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:16.255621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:14:16.255621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:14:16.255621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:14:16.255621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:14:16.256539 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:14:16.256539 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:14:16.256539 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:14:16.256539 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:14:16.256539 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:16.256539 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:16.256539 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:14:16.256539 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:14:16.256539 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:14:16.256539 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:14:16.256539 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:14:16.256539 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:14:16.256539 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:14:16.256539 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:14:16.256539 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:14:16.256539 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:14:16.257392 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:14:16.257392 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:14:16.257392 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:14:16.257392 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:14:16.257392 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:14:16.257392 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:14:16.257392 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:14:16.257392 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:14:16.257392 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:14:16.257392 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:14:16.257392 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 14:14:16.257392 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:14:16.259135 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 518/1032] 10.222.0.52 () {56 vars in 1120 bytes} [Sat Sep 24 14:14:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:14:16.262778 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb26bd-355f-4e9a-a125-e420bfaa4555 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:16.357970 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:16.362732 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb26bd-355f-4e9a-a125-e420bfaa4555 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:16.369675 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f969ec13-c80c-4939-b148-1d2ebb5d71fc None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:16.399543 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5af656-55c6-4959-aba1-ec6d8f490f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:16.400383 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5af656-55c6-4959-aba1-ec6d8f490f71 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:16.400666 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5af656-55c6-4959-aba1-ec6d8f490f71 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:16.400941 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5af656-55c6-4959-aba1-ec6d8f490f71 None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:16.401613 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5af656-55c6-4959-aba1-ec6d8f490f71 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:16.401916 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5af656-55c6-4959-aba1-ec6d8f490f71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:16.403492 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5af656-55c6-4959-aba1-ec6d8f490f71 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:16.418162 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 515/1033] 10.222.0.52 () {56 vars in 1115 bytes} [Sat Sep 24 14:14:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:16.418974 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5af656-55c6-4959-aba1-ec6d8f490f71 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:16.466568 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:16.477619 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f969ec13-c80c-4939-b148-1d2ebb5d71fc None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:16.511886 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead92b3d-c834-4d4f-8a7f-2bc28e692078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:16.512519 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead92b3d-c834-4d4f-8a7f-2bc28e692078 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:16.512847 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead92b3d-c834-4d4f-8a7f-2bc28e692078 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:16.513059 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead92b3d-c834-4d4f-8a7f-2bc28e692078 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:16.514381 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead92b3d-c834-4d4f-8a7f-2bc28e692078 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:16.515840 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead92b3d-c834-4d4f-8a7f-2bc28e692078 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:16.517408 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 519/1034] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:14:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:14:16.523224 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead92b3d-c834-4d4f-8a7f-2bc28e692078 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:16.540719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5af656-55c6-4959-aba1-ec6d8f490f71 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:16.574567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c1e75-04ba-4120-8768-7708d397c8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:16.575228 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c1e75-04ba-4120-8768-7708d397c8b5 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:16.575509 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c1e75-04ba-4120-8768-7708d397c8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:16.575780 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c1e75-04ba-4120-8768-7708d397c8b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:16.587678 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812c1e75-04ba-4120-8768-7708d397c8b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:16.589116 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812c1e75-04ba-4120-8768-7708d397c8b5 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:16.604746 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-812c1e75-04ba-4120-8768-7708d397c8b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:14:16.604746 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:16.604746 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:16.604746 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:16.604746 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:16.604746 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:16.604746 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:16.604746 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:16.604746 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:16.604746 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:16.604746 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:16.604746 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:16.604746 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:14:16.604746 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:14:16.604746 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:14:16.604746 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:14:16.605788 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:16.605788 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:16.605788 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:14:16.605788 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:14:16.605788 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:14:16.605788 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:14:16.605788 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:14:16.605788 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:14:16.605788 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:14:16.605788 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:14:16.605788 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:14:16.605788 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:14:16.605788 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:14:16.605788 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:14:16.605788 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:14:16.605788 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:14:16.605788 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:14:16.607078 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:14:16.607078 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:14:16.607078 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:16.607078 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 516/1035] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:14:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:14:16.607078 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3ffb4-09fb-4554-bb71-9a73b84c6ec7 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:16.607441 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3ffb4-09fb-4554-bb71-9a73b84c6ec7 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:16.607523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3ffb4-09fb-4554-bb71-9a73b84c6ec7 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:16.608757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 517/1036] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:14:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:14:16.611814 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3ffb4-09fb-4554-bb71-9a73b84c6ec7 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:16.625910 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead92b3d-c834-4d4f-8a7f-2bc28e692078 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:16.626525 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:16.660258 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d09f117-7787-480f-9a7f-0cb73a585edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:16.660853 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09f117-7787-480f-9a7f-0cb73a585edf None admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:16.661126 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09f117-7787-480f-9a7f-0cb73a585edf None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:16.661389 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09f117-7787-480f-9a7f-0cb73a585edf None admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:16.662029 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d09f117-7787-480f-9a7f-0cb73a585edf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:16.663479 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d09f117-7787-480f-9a7f-0cb73a585edf None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:16.686220 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 520/1037] 10.222.0.52 () {60 vars in 1132 bytes} [Sat Sep 24 14:14:16 2022] POST /identity/v3/users => generated 285 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:14:16.687517 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ddb72-93df-49b8-8388-c44afef2433b None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:16.687799 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ddb72-93df-49b8-8388-c44afef2433b None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:16.688067 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ddb72-93df-49b8-8388-c44afef2433b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:16.707194 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-7a6ddb72-93df-49b8-8388-c44afef2433b None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:14:16.708443 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-7a6ddb72-93df-49b8-8388-c44afef2433b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:14:16.719853 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:16.720114 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3ffb4-09fb-4554-bb71-9a73b84c6ec7 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:16.772925 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5575f5-5dd3-4e22-8d44-44a36e64283a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:16.774464 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5575f5-5dd3-4e22-8d44-44a36e64283a None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:16.774464 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5575f5-5dd3-4e22-8d44-44a36e64283a None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:16.774713 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5575f5-5dd3-4e22-8d44-44a36e64283a None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:16.775308 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5575f5-5dd3-4e22-8d44-44a36e64283a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:16.775612 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5575f5-5dd3-4e22-8d44-44a36e64283a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:16.777000 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5575f5-5dd3-4e22-8d44-44a36e64283a None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:16.794934 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 518/1038] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:14:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:14:16.795900 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5575f5-5dd3-4e22-8d44-44a36e64283a None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:16.797196 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 521/1039] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:14:16 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:14:16.882674 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:16.914073 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5575f5-5dd3-4e22-8d44-44a36e64283a None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:16.949323 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f35ff-003f-4ced-bce7-ae8f454429b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:16.950014 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f35ff-003f-4ced-bce7-ae8f454429b2 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:16.950293 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f35ff-003f-4ced-bce7-ae8f454429b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:16.950557 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f35ff-003f-4ced-bce7-ae8f454429b2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:16.958179 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f35ff-003f-4ced-bce7-ae8f454429b2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:16.959687 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f35ff-003f-4ced-bce7-ae8f454429b2 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:16.971058 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-f50f35ff-003f-4ced-bce7-ae8f454429b2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 14:14:16.971058 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:16.971058 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:16.971058 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:16.971058 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:16.971058 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:16.971058 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:16.971058 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:16.971058 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:16.971058 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:16.971058 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:16.971058 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:16.971058 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:14:16.971058 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:14:16.971058 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:14:16.971058 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:14:16.971992 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:16.971992 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:16.971992 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:14:16.971992 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:14:16.971992 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:14:16.971992 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:14:16.971992 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:14:16.971992 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:14:16.971992 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:14:16.971992 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:14:16.971992 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:14:16.971992 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:14:16.971992 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:14:16.971992 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:14:16.971992 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:14:16.971992 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:14:16.971992 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:14:16.971992 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:14:16.972992 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:14:16.972992 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:14:16.972992 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:14:16.972992 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:14:16.972992 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:14:16.972992 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:14:16.972992 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:14:16.972992 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:14:16.972992 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 14:14:16.972992 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:16.972992 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 519/1040] 10.222.0.52 () {56 vars in 1093 bytes} [Sat Sep 24 14:14:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:14:16.980110 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ddb72-93df-49b8-8388-c44afef2433b None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:17.068694 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f35ff-003f-4ced-bce7-ae8f454429b2 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:17.093516 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ddb72-93df-49b8-8388-c44afef2433b None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:17.129710 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826fa8b5-4995-46f2-a767-f889386068f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:17.130576 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826fa8b5-4995-46f2-a767-f889386068f9 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:17.130576 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826fa8b5-4995-46f2-a767-f889386068f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:17.130862 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826fa8b5-4995-46f2-a767-f889386068f9 None admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:17.131593 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826fa8b5-4995-46f2-a767-f889386068f9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:17.131894 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826fa8b5-4995-46f2-a767-f889386068f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:17.133884 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826fa8b5-4995-46f2-a767-f889386068f9 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:17.155614 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 522/1041] 10.222.0.52 () {56 vars in 1133 bytes} [Sat Sep 24 14:14:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:17.156629 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826fa8b5-4995-46f2-a767-f889386068f9 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:17.159921 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:17.182400 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f35ff-003f-4ced-bce7-ae8f454429b2 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:17.213594 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:17.219750 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f6515-697d-455b-8d52-d1bfd8f040bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:17.220378 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f6515-697d-455b-8d52-d1bfd8f040bb None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:17.220650 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f6515-697d-455b-8d52-d1bfd8f040bb None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:17.220914 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f6515-697d-455b-8d52-d1bfd8f040bb None admin] PATH_INFO: `/v3/services/image` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:17.221489 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0f6515-697d-455b-8d52-d1bfd8f040bb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:17.222993 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0f6515-697d-455b-8d52-d1bfd8f040bb None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:17.233057 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-7e0f6515-697d-455b-8d52-d1bfd8f040bb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 14:14:17.233057 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:17.233057 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:14:17.233057 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:14:17.233057 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:14:17.233057 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:14:17.233057 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:14:17.233057 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:14:17.233057 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 14:14:17.233057 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:17.233057 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:14:17.233057 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:17.233057 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:17.233057 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:17.233057 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:17.233057 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:17.233057 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:17.234037 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:17.234037 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:17.234037 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:17.234037 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:17.234037 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:17.234037 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:17.234037 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:14:17.234037 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:14:17.234037 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:14:17.234037 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:14:17.234037 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:17.234037 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:17.234037 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:14:17.234037 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:14:17.234037 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:14:17.234037 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:14:17.235059 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:14:17.235059 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:14:17.235059 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:14:17.235059 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:14:17.235059 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:14:17.235059 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:14:17.235059 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:14:17.235059 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:14:17.235059 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:14:17.235059 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:14:17.235059 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:14:17.235059 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:14:17.235059 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 14:14:17.235059 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:17.235059 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 520/1042] 10.222.0.52 () {56 vars in 1105 bytes} [Sat Sep 24 14:14:17 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:14:17.239935 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f6515-697d-455b-8d52-d1bfd8f040bb None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:17.272295 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826fa8b5-4995-46f2-a767-f889386068f9 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:17.308140 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87367fec-23d3-4883-813c-9457b9a718b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:17.308749 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87367fec-23d3-4883-813c-9457b9a718b8 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:17.309024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87367fec-23d3-4883-813c-9457b9a718b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:17.309296 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87367fec-23d3-4883-813c-9457b9a718b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:17.345342 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f6515-697d-455b-8d52-d1bfd8f040bb None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:17.379862 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87367fec-23d3-4883-813c-9457b9a718b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:17.381299 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87367fec-23d3-4883-813c-9457b9a718b8 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:17.382684 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cd0ac3-79cb-4f70-b75a-577b5f80f956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:17.383262 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:17.384776 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd0ac3-79cb-4f70-b75a-577b5f80f956 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:17.384985 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd0ac3-79cb-4f70-b75a-577b5f80f956 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:17.385967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd0ac3-79cb-4f70-b75a-577b5f80f956 None admin] PATH_INFO: `/v3/services` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:17.386648 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cd0ac3-79cb-4f70-b75a-577b5f80f956 None admin] RBAC: Adding query filter params (name=image) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:17.386929 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cd0ac3-79cb-4f70-b75a-577b5f80f956 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:17.388277 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cd0ac3-79cb-4f70-b75a-577b5f80f956 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:17.394275 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 521/1043] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:14:17 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:14:17.400164 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cd0ac3-79cb-4f70-b75a-577b5f80f956 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:17.465397 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:17.490364 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 523/1044] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:14:17 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 24 14:14:17.491532 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87367fec-23d3-4883-813c-9457b9a718b8 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:17.514557 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cd0ac3-79cb-4f70-b75a-577b5f80f956 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:17.553316 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeb6eb3-48f6-4cb6-baf1-366bbf63a57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:17.553970 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb6eb3-48f6-4cb6-baf1-366bbf63a57a None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:17.554245 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb6eb3-48f6-4cb6-baf1-366bbf63a57a None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:17.554562 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb6eb3-48f6-4cb6-baf1-366bbf63a57a None admin] PATH_INFO: `/v3/services` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:17.555846 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeb6eb3-48f6-4cb6-baf1-366bbf63a57a None admin] RBAC: Adding query filter params (type=image) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:17.556259 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeb6eb3-48f6-4cb6-baf1-366bbf63a57a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:17.557730 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeb6eb3-48f6-4cb6-baf1-366bbf63a57a None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:17.563262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 522/1045] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:14:17 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:14:17.568602 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeb6eb3-48f6-4cb6-baf1-366bbf63a57a None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:17.589782 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87367fec-23d3-4883-813c-9457b9a718b8 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:17.593909 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:17.631291 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c6fe42-51d7-4e3f-855b-95e79c8c9683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:17.631930 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c6fe42-51d7-4e3f-855b-95e79c8c9683 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:17.632203 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c6fe42-51d7-4e3f-855b-95e79c8c9683 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:17.632475 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c6fe42-51d7-4e3f-855b-95e79c8c9683 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:17.643413 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c6fe42-51d7-4e3f-855b-95e79c8c9683 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:17.644902 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c6fe42-51d7-4e3f-855b-95e79c8c9683 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:17.661673 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-e3c6fe42-51d7-4e3f-855b-95e79c8c9683 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:14:17.661673 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:17.661673 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:17.661673 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:17.661673 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:17.661673 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:17.661673 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:17.661673 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:17.661673 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:17.661673 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:17.661673 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:17.661673 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:17.661673 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:14:17.661673 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:14:17.661673 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:14:17.661673 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:14:17.662748 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:17.662748 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:17.662748 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:14:17.662748 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:14:17.662748 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:14:17.662748 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:14:17.662748 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:14:17.662748 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:14:17.662748 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:14:17.662748 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:14:17.662748 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:14:17.662748 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:14:17.662748 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:14:17.662748 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:14:17.662748 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:14:17.662748 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:14:17.662748 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:14:17.663697 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:14:17.663697 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:14:17.663697 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:14:17.667637 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 524/1046] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:14:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:14:17.669506 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c6fe42-51d7-4e3f-855b-95e79c8c9683 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:17.677225 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeb6eb3-48f6-4cb6-baf1-366bbf63a57a None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:17.703828 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:17.717608 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19af1bc6-5445-4a13-86a6-c034f202a981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:17.718268 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af1bc6-5445-4a13-86a6-c034f202a981 None admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:17.718546 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af1bc6-5445-4a13-86a6-c034f202a981 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:17.718830 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af1bc6-5445-4a13-86a6-c034f202a981 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:17.719491 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19af1bc6-5445-4a13-86a6-c034f202a981 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:17.720851 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19af1bc6-5445-4a13-86a6-c034f202a981 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:17.744249 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 523/1047] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:14:17 2022] POST /identity/v3/endpoints => generated 327 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:14:17.779024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c6fe42-51d7-4e3f-855b-95e79c8c9683 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:17.825454 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f194f-11a3-4cca-87e9-1c7249330dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:17.826102 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f194f-11a3-4cca-87e9-1c7249330dad None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:17.826337 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f194f-11a3-4cca-87e9-1c7249330dad None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:17.826603 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f194f-11a3-4cca-87e9-1c7249330dad None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:17.827288 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9f194f-11a3-4cca-87e9-1c7249330dad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:17.827588 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9f194f-11a3-4cca-87e9-1c7249330dad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:17.828978 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9f194f-11a3-4cca-87e9-1c7249330dad None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:17.846603 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 525/1048] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:14:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:17.853156 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19af1bc6-5445-4a13-86a6-c034f202a981 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:17.961208 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19af1bc6-5445-4a13-86a6-c034f202a981 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:17.996703 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8196aeeb-58c7-4766-8e60-fa267337ef3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:17.997333 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196aeeb-58c7-4766-8e60-fa267337ef3c None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:17.997608 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196aeeb-58c7-4766-8e60-fa267337ef3c None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:17.997876 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196aeeb-58c7-4766-8e60-fa267337ef3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:18.004812 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8196aeeb-58c7-4766-8e60-fa267337ef3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:18.006242 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8196aeeb-58c7-4766-8e60-fa267337ef3c None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:18.017530 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-8196aeeb-58c7-4766-8e60-fa267337ef3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:14:18.017530 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:18.017530 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:18.017530 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:18.017530 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:18.017530 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:18.017530 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:18.017530 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:18.017530 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:18.017530 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:18.017530 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:18.017530 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:18.017530 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:14:18.017530 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:14:18.017530 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:14:18.018417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:14:18.018417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:18.018417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:18.018417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:14:18.018417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:14:18.018417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:14:18.018417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:14:18.018417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:14:18.018417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:14:18.018417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:14:18.018417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:14:18.018417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:14:18.018417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:14:18.018417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:14:18.018417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:14:18.018417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:14:18.018417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:14:18.019330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:14:18.019330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:14:18.019330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:14:18.019330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:14:18.019330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:14:18.019330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:14:18.019330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:14:18.019330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:18.019330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 524/1049] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:14:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:14:18.026766 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f194f-11a3-4cca-87e9-1c7249330dad None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:18.040033 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:18.132013 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f194f-11a3-4cca-87e9-1c7249330dad None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:18.166566 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb62a440-6586-4c00-96ce-cb9cbd6c3d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:18.167196 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62a440-6586-4c00-96ce-cb9cbd6c3d38 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:18.167470 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62a440-6586-4c00-96ce-cb9cbd6c3d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:18.167734 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62a440-6586-4c00-96ce-cb9cbd6c3d38 None admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:18.168413 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb62a440-6586-4c00-96ce-cb9cbd6c3d38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:18.168709 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb62a440-6586-4c00-96ce-cb9cbd6c3d38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:18.170126 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb62a440-6586-4c00-96ce-cb9cbd6c3d38 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:18.181539 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 526/1050] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 14:14:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:14:18.187481 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8196aeeb-58c7-4766-8e60-fa267337ef3c None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:18.234965 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:18.306706 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8196aeeb-58c7-4766-8e60-fa267337ef3c None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:18.342500 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39ec1e5-ff75-4575-a15e-0005de197834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:18.343475 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ec1e5-ff75-4575-a15e-0005de197834 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:18.344057 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ec1e5-ff75-4575-a15e-0005de197834 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:18.344616 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ec1e5-ff75-4575-a15e-0005de197834 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:18.345577 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39ec1e5-ff75-4575-a15e-0005de197834 None admin] RBAC: Adding query filter params (role.id=30791f3f2a7841d2a80aab92cf4f23f5, scope.project.id=36a3ae7bab25471fb989136150b6438c, user.id=79eaaae97aac4282a613939e5b768376) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:18.346155 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39ec1e5-ff75-4575-a15e-0005de197834 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:18.347829 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39ec1e5-ff75-4575-a15e-0005de197834 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:18.354047 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 525/1051] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:14:18 2022] GET /identity/v3/role_assignments?user.id=79eaaae97aac4282a613939e5b768376&scope.project.id=36a3ae7bab25471fb989136150b6438c&role.id=30791f3f2a7841d2a80aab92cf4f23f5 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:18.944178 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffb9023-09d5-462a-b855-0094852355ed None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:18.944728 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffb9023-09d5-462a-b855-0094852355ed None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:18.944728 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffb9023-09d5-462a-b855-0094852355ed None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:18.946216 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 527/1052] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:14:18.951270 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf7389-175a-4b11-a4b2-c7de0aee6005 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:18.951686 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf7389-175a-4b11-a4b2-c7de0aee6005 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:18.951847 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf7389-175a-4b11-a4b2-c7de0aee6005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:18.969784 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-4abf7389-175a-4b11-a4b2-c7de0aee6005 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:14:18.971037 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-4abf7389-175a-4b11-a4b2-c7de0aee6005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:14:19.041096 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 526/1053] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:14:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:14:19.314185 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffb9023-09d5-462a-b855-0094852355ed None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:19.385797 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:19.418322 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffb9023-09d5-462a-b855-0094852355ed None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:19.453349 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37dd5f-98a2-4b5b-ab08-23bc3279115e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:19.453985 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37dd5f-98a2-4b5b-ab08-23bc3279115e None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:19.454264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37dd5f-98a2-4b5b-ab08-23bc3279115e None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:19.454535 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37dd5f-98a2-4b5b-ab08-23bc3279115e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:19.465202 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e37dd5f-98a2-4b5b-ab08-23bc3279115e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:14:19.469969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-0e37dd5f-98a2-4b5b-ab08-23bc3279115e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:14:19.469969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:19.469969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:19.469969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:19.469969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:19.469969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:19.469969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:19.469969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:19.469969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:19.469969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:19.469969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:19.469969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:19.469969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:14:19.469969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:14:19.469969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:14:19.469969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:14:19.471156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:14:19.471156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:14:19.471156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:14:19.471156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:14:19.471156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:19.471156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:19.471156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:14:19.471156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:14:19.471156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:14:19.471156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:14:19.471156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:14:19.471156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:14:19.471156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:14:19.471156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:14:19.471156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:14:19.471156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:14:19.471990 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:14:19.471990 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:14:19.471990 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:14:19.471990 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:14:19.471990 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:14:19.471990 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:14:19.471990 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:14:19.471990 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:14:19.471990 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:14:19.471990 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:14:19.471990 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:14:19.471990 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:14:19.475302 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 528/1054] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 14:14:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:14:19.477014 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf7389-175a-4b11-a4b2-c7de0aee6005 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:19.511646 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:19.581156 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf7389-175a-4b11-a4b2-c7de0aee6005 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:19.614151 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d201b29-44a3-4bd8-b26e-6ccbc3fd20af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:19.614754 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d201b29-44a3-4bd8-b26e-6ccbc3fd20af None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:19.615055 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d201b29-44a3-4bd8-b26e-6ccbc3fd20af None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:19.615322 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d201b29-44a3-4bd8-b26e-6ccbc3fd20af None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:19.615977 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d201b29-44a3-4bd8-b26e-6ccbc3fd20af None admin] RBAC: Adding query filter params (name=admin) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:19.616273 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d201b29-44a3-4bd8-b26e-6ccbc3fd20af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:19.618129 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d201b29-44a3-4bd8-b26e-6ccbc3fd20af None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:19.630776 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 527/1055] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:14:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:19.631998 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37dd5f-98a2-4b5b-ab08-23bc3279115e None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:19.731161 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37dd5f-98a2-4b5b-ab08-23bc3279115e None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:19.758735 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:19.765962 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484350b5-2166-42bd-b3c9-826dc6d3008c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:19.766599 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484350b5-2166-42bd-b3c9-826dc6d3008c None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:19.766901 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484350b5-2166-42bd-b3c9-826dc6d3008c None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:19.767173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484350b5-2166-42bd-b3c9-826dc6d3008c None admin] PATH_INFO: `/v3/users/3dd3045794324575872b0719cd534e3b` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:19.781311 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484350b5-2166-42bd-b3c9-826dc6d3008c None admin] RBAC: Authorizing `identity:get_user(user_id=3dd3045794324575872b0719cd534e3b)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:19.782734 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484350b5-2166-42bd-b3c9-826dc6d3008c None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:19.784781 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 529/1056] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:14:19 2022] GET /identity/v3/users/3dd3045794324575872b0719cd534e3b => generated 285 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:14:19.790428 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d201b29-44a3-4bd8-b26e-6ccbc3fd20af None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:19.899660 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:19.906383 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d201b29-44a3-4bd8-b26e-6ccbc3fd20af None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:19.940139 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99362287-b82a-4770-ac73-a73dfde8b62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:19.940894 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99362287-b82a-4770-ac73-a73dfde8b62e None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:19.941269 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99362287-b82a-4770-ac73-a73dfde8b62e None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:19.941637 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99362287-b82a-4770-ac73-a73dfde8b62e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:19.957780 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea5999-c0d6-4c5a-bf2c-84b5ed00182c None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:19.958302 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea5999-c0d6-4c5a-bf2c-84b5ed00182c None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:19.958302 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea5999-c0d6-4c5a-bf2c-84b5ed00182c None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:19.959694 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 530/1057] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:14:19.965234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd5f2d-ae0d-48de-bb4d-60b6a5f2e743 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:19.965518 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd5f2d-ae0d-48de-bb4d-60b6a5f2e743 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:19.965796 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd5f2d-ae0d-48de-bb4d-60b6a5f2e743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:19.983457 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-f7fd5f2d-ae0d-48de-bb4d-60b6a5f2e743 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:14:19.984644 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-f7fd5f2d-ae0d-48de-bb4d-60b6a5f2e743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:14:20.015355 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99362287-b82a-4770-ac73-a73dfde8b62e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:20.016800 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99362287-b82a-4770-ac73-a73dfde8b62e None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:20.056988 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 531/1058] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:14:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:14:20.117370 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:20.125145 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 528/1059] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:14:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 14:14:20.130585 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd5f2d-ae0d-48de-bb4d-60b6a5f2e743 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:20.231094 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd5f2d-ae0d-48de-bb4d-60b6a5f2e743 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:20.248628 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:20.267019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d57b88d-3b9f-49bc-8ce3-ab45188ebf88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:20.267646 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57b88d-3b9f-49bc-8ce3-ab45188ebf88 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:20.267921 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57b88d-3b9f-49bc-8ce3-ab45188ebf88 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:20.268188 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57b88d-3b9f-49bc-8ce3-ab45188ebf88 None admin] PATH_INFO: `/v3/projects/c7c889300bce4c7e9bddec70ac0d5aa1` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:20.269339 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d57b88d-3b9f-49bc-8ce3-ab45188ebf88 None admin] RBAC: Authorizing `identity:get_project(project_id=c7c889300bce4c7e9bddec70ac0d5aa1)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:20.270755 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d57b88d-3b9f-49bc-8ce3-ab45188ebf88 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:20.276904 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 532/1060] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:14:20 2022] GET /identity/v3/projects/c7c889300bce4c7e9bddec70ac0d5aa1 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:20.278950 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99362287-b82a-4770-ac73-a73dfde8b62e None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:20.327956 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d57b88d-3b9f-49bc-8ce3-ab45188ebf88 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:20.387300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99362287-b82a-4770-ac73-a73dfde8b62e None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:20.428302 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe8a690-dc0d-48c1-8433-53dd6d6afe43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:20.428917 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8a690-dc0d-48c1-8433-53dd6d6afe43 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:20.429197 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8a690-dc0d-48c1-8433-53dd6d6afe43 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:20.429469 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8a690-dc0d-48c1-8433-53dd6d6afe43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:20.430807 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe8a690-dc0d-48c1-8433-53dd6d6afe43 None admin] RBAC: Adding query filter params (role.id=d00b78c32dbf4cc7925a55bb621d8fcf, scope.project.id=c7c889300bce4c7e9bddec70ac0d5aa1, user.id=3dd3045794324575872b0719cd534e3b) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:20.431365 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe8a690-dc0d-48c1-8433-53dd6d6afe43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:20.432558 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe8a690-dc0d-48c1-8433-53dd6d6afe43 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:20.437488 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d57b88d-3b9f-49bc-8ce3-ab45188ebf88 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:20.443058 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 529/1061] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:14:20 2022] GET /identity/v3/role_assignments?user.id=3dd3045794324575872b0719cd534e3b&scope.project.id=c7c889300bce4c7e9bddec70ac0d5aa1&role.id=d00b78c32dbf4cc7925a55bb621d8fcf => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:20.487808 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42642f4-7a34-4764-9621-91c17bc2d9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:20.488461 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42642f4-7a34-4764-9621-91c17bc2d9e0 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:20.488741 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42642f4-7a34-4764-9621-91c17bc2d9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:20.489015 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42642f4-7a34-4764-9621-91c17bc2d9e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:20.530409 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ceda7f-ab38-41d9-9547-de1cbcc5342e None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:20.530697 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ceda7f-ab38-41d9-9547-de1cbcc5342e None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:20.531025 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ceda7f-ab38-41d9-9547-de1cbcc5342e None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:20.532404 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 530/1062] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:14:20.538698 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b820e-856d-48cb-903e-bb2d97921e82 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:20.539018 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b820e-856d-48cb-903e-bb2d97921e82 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:20.539916 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b820e-856d-48cb-903e-bb2d97921e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:20.554785 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:20.560680 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-222b820e-856d-48cb-903e-bb2d97921e82 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:14:20.561917 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-222b820e-856d-48cb-903e-bb2d97921e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:14:20.572310 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42642f4-7a34-4764-9621-91c17bc2d9e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:20.574556 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42642f4-7a34-4764-9621-91c17bc2d9e0 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:20.625097 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:20.638926 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 531/1063] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:14:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:14:20.690856 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 533/1064] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:14:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 24 14:14:20.695519 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222b820e-856d-48cb-903e-bb2d97921e82 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:20.795233 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222b820e-856d-48cb-903e-bb2d97921e82 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:20.833406 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25759d2-c4a4-4dae-9b17-94f3ee97a2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:20.834462 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25759d2-c4a4-4dae-9b17-94f3ee97a2a8 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:20.834738 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25759d2-c4a4-4dae-9b17-94f3ee97a2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:20.835034 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25759d2-c4a4-4dae-9b17-94f3ee97a2a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:20.845896 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25759d2-c4a4-4dae-9b17-94f3ee97a2a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:20.847360 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25759d2-c4a4-4dae-9b17-94f3ee97a2a8 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:20.862532 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-d25759d2-c4a4-4dae-9b17-94f3ee97a2a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:14:20.862532 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:20.862532 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:20.862532 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:20.862532 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:20.862532 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:20.862532 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:20.862532 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:20.862532 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:20.862532 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:20.862532 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:20.862532 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:20.862532 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:14:20.862532 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:14:20.862532 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:14:20.862532 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:14:20.868587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:20.868587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:20.868587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:14:20.868587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:14:20.868587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:14:20.868587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:14:20.868587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:14:20.868587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:14:20.868587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:14:20.868587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:14:20.868587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:14:20.868587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:14:20.868587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:14:20.868587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:14:20.868587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:14:20.868587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:14:20.868587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:14:20.869509 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:14:20.869509 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:14:20.869509 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:20.869509 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 532/1065] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:14:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:14:20.869738 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42642f4-7a34-4764-9621-91c17bc2d9e0 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:20.915212 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25759d2-c4a4-4dae-9b17-94f3ee97a2a8 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:20.958338 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:20.962768 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:20.974910 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42642f4-7a34-4764-9621-91c17bc2d9e0 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:21.009991 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9318eb65-abed-4534-99dc-e46be254ad02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:21.010608 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318eb65-abed-4534-99dc-e46be254ad02 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:21.010920 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318eb65-abed-4534-99dc-e46be254ad02 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:21.011193 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318eb65-abed-4534-99dc-e46be254ad02 None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:21.011828 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9318eb65-abed-4534-99dc-e46be254ad02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:21.012128 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9318eb65-abed-4534-99dc-e46be254ad02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:21.013493 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9318eb65-abed-4534-99dc-e46be254ad02 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:21.022503 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25759d2-c4a4-4dae-9b17-94f3ee97a2a8 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:21.031226 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 534/1066] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:14:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:21.060401 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f92e0ec-32be-408d-9b9d-c4d4457529f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:21.061055 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92e0ec-32be-408d-9b9d-c4d4457529f3 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:21.061336 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92e0ec-32be-408d-9b9d-c4d4457529f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:21.061609 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92e0ec-32be-408d-9b9d-c4d4457529f3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:21.075641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f92e0ec-32be-408d-9b9d-c4d4457529f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:14:21.080829 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-0f92e0ec-32be-408d-9b9d-c4d4457529f3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 14:14:21.080829 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:21.080829 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:21.080829 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:21.080829 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:21.080829 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:21.080829 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:21.080829 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:21.080829 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:21.080829 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:21.080829 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:21.080829 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:21.080829 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:14:21.080829 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:14:21.080829 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:14:21.080829 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:14:21.081817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:14:21.081817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:14:21.081817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:14:21.081817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:14:21.081817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:21.081817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:21.081817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:14:21.081817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:14:21.081817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:14:21.081817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:14:21.081817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:14:21.081817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:14:21.081817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:14:21.081817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:14:21.081817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:14:21.081817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:14:21.082742 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:14:21.082742 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:14:21.082742 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:14:21.082742 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:14:21.082742 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:14:21.082742 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:14:21.082742 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:14:21.082742 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:14:21.082742 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:14:21.082742 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:14:21.082742 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 14:14:21.082742 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:21.087478 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 533/1067] 10.222.0.52 () {56 vars in 1120 bytes} [Sat Sep 24 14:14:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:14:21.088818 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9318eb65-abed-4534-99dc-e46be254ad02 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:21.197519 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9318eb65-abed-4534-99dc-e46be254ad02 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:21.238488 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25151774-67c9-40d7-a1e3-d2fbb1acd48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:21.239142 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25151774-67c9-40d7-a1e3-d2fbb1acd48a None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:21.239424 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25151774-67c9-40d7-a1e3-d2fbb1acd48a None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:21.239697 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25151774-67c9-40d7-a1e3-d2fbb1acd48a None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:21.240364 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25151774-67c9-40d7-a1e3-d2fbb1acd48a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:21.240665 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25151774-67c9-40d7-a1e3-d2fbb1acd48a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:21.242029 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25151774-67c9-40d7-a1e3-d2fbb1acd48a None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:21.251851 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 535/1068] 10.222.0.52 () {56 vars in 1115 bytes} [Sat Sep 24 14:14:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:21.257494 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f92e0ec-32be-408d-9b9d-c4d4457529f3 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:21.365682 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f92e0ec-32be-408d-9b9d-c4d4457529f3 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:21.367271 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:21.400986 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1bb2d-7559-4702-b2a1-0301e8622d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:21.401612 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1bb2d-7559-4702-b2a1-0301e8622d32 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:21.401891 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1bb2d-7559-4702-b2a1-0301e8622d32 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:21.402164 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1bb2d-7559-4702-b2a1-0301e8622d32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:21.412171 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce1bb2d-7559-4702-b2a1-0301e8622d32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:21.413600 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce1bb2d-7559-4702-b2a1-0301e8622d32 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:21.428181 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-7ce1bb2d-7559-4702-b2a1-0301e8622d32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:14:21.428181 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:21.428181 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:21.428181 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:21.428181 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:21.428181 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:21.428181 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:21.428181 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:21.428181 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:21.428181 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:21.428181 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:21.428181 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:21.428181 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:14:21.428181 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:14:21.428181 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:14:21.428181 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:14:21.429269 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:21.429269 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:21.429269 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:14:21.429269 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:14:21.429269 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:14:21.429269 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:14:21.429269 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:14:21.429269 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:14:21.429269 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:14:21.429269 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:14:21.429269 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:14:21.429269 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:14:21.429269 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:14:21.429269 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:14:21.429269 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:14:21.429269 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:14:21.429269 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:14:21.431125 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:14:21.431125 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:14:21.431125 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:21.431125 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 534/1069] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:14:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:14:21.435912 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25151774-67c9-40d7-a1e3-d2fbb1acd48a None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:21.446582 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:21.536338 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25151774-67c9-40d7-a1e3-d2fbb1acd48a None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:21.569588 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e1b5b-f146-4c1f-b083-dac1d9bf9f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:21.570191 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e1b5b-f146-4c1f-b083-dac1d9bf9f0d None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:21.570488 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e1b5b-f146-4c1f-b083-dac1d9bf9f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:21.570744 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e1b5b-f146-4c1f-b083-dac1d9bf9f0d None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:21.571408 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e1b5b-f146-4c1f-b083-dac1d9bf9f0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:21.571709 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e1b5b-f146-4c1f-b083-dac1d9bf9f0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:21.573073 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e1b5b-f146-4c1f-b083-dac1d9bf9f0d None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:21.583907 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 536/1070] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:14:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:21.590035 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1bb2d-7559-4702-b2a1-0301e8622d32 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:21.687914 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1bb2d-7559-4702-b2a1-0301e8622d32 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:21.720551 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a17773-420b-4e65-a59d-c14d9382fcc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:21.721177 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a17773-420b-4e65-a59d-c14d9382fcc9 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:21.721455 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a17773-420b-4e65-a59d-c14d9382fcc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:21.721726 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a17773-420b-4e65-a59d-c14d9382fcc9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:21.729582 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a17773-420b-4e65-a59d-c14d9382fcc9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:21.731039 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a17773-420b-4e65-a59d-c14d9382fcc9 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:21.741818 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-b7a17773-420b-4e65-a59d-c14d9382fcc9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 14:14:21.741818 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:21.741818 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:21.741818 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:21.741818 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:21.741818 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:21.741818 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:21.741818 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:21.741818 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:21.741818 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:21.741818 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:21.741818 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:21.741818 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:14:21.741818 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:14:21.741818 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:14:21.741818 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:14:21.742980 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:21.742980 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:21.742980 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:14:21.742980 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:14:21.742980 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:14:21.742980 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:14:21.742980 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:14:21.742980 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:14:21.742980 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:14:21.742980 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:14:21.742980 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:14:21.742980 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:14:21.742980 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:14:21.742980 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:14:21.742980 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:14:21.742980 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:14:21.742980 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:14:21.742980 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:14:21.744046 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:14:21.744046 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:14:21.744046 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:14:21.744046 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:14:21.744046 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:14:21.744046 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:14:21.744046 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:14:21.744046 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:14:21.744046 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 14:14:21.744046 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:21.744046 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 535/1071] 10.222.0.52 () {56 vars in 1093 bytes} [Sat Sep 24 14:14:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:14:21.748819 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e1b5b-f146-4c1f-b083-dac1d9bf9f0d None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:21.813604 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:21.849092 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e1b5b-f146-4c1f-b083-dac1d9bf9f0d None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:21.881718 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a62966-a4fb-4362-8aa0-ed10e07026b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:21.882319 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a62966-a4fb-4362-8aa0-ed10e07026b1 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:21.882594 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a62966-a4fb-4362-8aa0-ed10e07026b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:21.882884 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a62966-a4fb-4362-8aa0-ed10e07026b1 None admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:21.883606 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a62966-a4fb-4362-8aa0-ed10e07026b1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:21.883909 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a62966-a4fb-4362-8aa0-ed10e07026b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:21.885333 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a62966-a4fb-4362-8aa0-ed10e07026b1 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:21.902246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 537/1072] 10.222.0.52 () {56 vars in 1133 bytes} [Sat Sep 24 14:14:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:21.907961 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a17773-420b-4e65-a59d-c14d9382fcc9 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:21.924509 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:22.007627 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a17773-420b-4e65-a59d-c14d9382fcc9 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:22.040494 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236547bd-5cc5-42c1-83ba-b695fd879ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:22.041126 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236547bd-5cc5-42c1-83ba-b695fd879ae1 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:22.041404 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236547bd-5cc5-42c1-83ba-b695fd879ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:22.041670 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236547bd-5cc5-42c1-83ba-b695fd879ae1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:22.052372 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236547bd-5cc5-42c1-83ba-b695fd879ae1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:22.053798 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236547bd-5cc5-42c1-83ba-b695fd879ae1 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:22.067504 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-236547bd-5cc5-42c1-83ba-b695fd879ae1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:14:22.067504 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:22.067504 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:22.067504 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:22.067504 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:22.067504 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:22.067504 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:22.067504 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:22.067504 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:22.067504 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:22.067504 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:22.067504 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:22.067504 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:14:22.067504 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:14:22.067504 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:14:22.067504 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:14:22.068580 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:22.068580 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:22.068580 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:14:22.068580 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:14:22.068580 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:14:22.068580 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:14:22.068580 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:14:22.068580 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:14:22.068580 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:14:22.068580 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:14:22.068580 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:14:22.068580 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:14:22.068580 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:14:22.068580 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:14:22.068580 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:14:22.068580 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:14:22.068580 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:14:22.069473 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:14:22.069473 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:14:22.069473 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:22.069473 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 536/1073] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:14:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:14:22.074704 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a62966-a4fb-4362-8aa0-ed10e07026b1 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:22.171309 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a62966-a4fb-4362-8aa0-ed10e07026b1 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:22.196453 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:22.205441 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a2a659-51cc-462f-bda1-0e341f39357f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:22.206045 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2a659-51cc-462f-bda1-0e341f39357f None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:22.206325 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2a659-51cc-462f-bda1-0e341f39357f None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:22.206590 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2a659-51cc-462f-bda1-0e341f39357f None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:22.207253 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a2a659-51cc-462f-bda1-0e341f39357f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:22.207551 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a2a659-51cc-462f-bda1-0e341f39357f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:22.208904 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a2a659-51cc-462f-bda1-0e341f39357f None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:22.220735 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 538/1074] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:14:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:22.226086 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236547bd-5cc5-42c1-83ba-b695fd879ae1 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:22.288681 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:22.324867 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236547bd-5cc5-42c1-83ba-b695fd879ae1 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:22.356667 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c3433-8301-436f-a603-fc5ae33f77ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:22.357285 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c3433-8301-436f-a603-fc5ae33f77ed None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:22.357562 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c3433-8301-436f-a603-fc5ae33f77ed None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:22.357836 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c3433-8301-436f-a603-fc5ae33f77ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:22.364293 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c3433-8301-436f-a603-fc5ae33f77ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:22.365711 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c3433-8301-436f-a603-fc5ae33f77ed None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:22.377147 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-7d6c3433-8301-436f-a603-fc5ae33f77ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:14:22.377147 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:22.377147 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:22.377147 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:22.377147 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:22.377147 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:22.377147 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:22.377147 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:22.377147 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:22.377147 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:22.377147 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:22.377147 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:22.377147 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:14:22.377147 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:14:22.377147 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:14:22.378001 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:14:22.378001 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:22.378001 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:22.378001 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:14:22.378001 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:14:22.378001 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:14:22.378001 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:14:22.378001 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:14:22.378001 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:14:22.378001 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:14:22.378001 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:14:22.378001 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:14:22.378001 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:14:22.378001 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:14:22.378001 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:14:22.378001 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:14:22.378001 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:14:22.378904 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:14:22.378904 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:14:22.378904 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:14:22.378904 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:14:22.378904 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:14:22.378904 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:14:22.378904 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:14:22.378904 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:22.382958 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 537/1075] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:14:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:14:22.384461 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a2a659-51cc-462f-bda1-0e341f39357f None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:22.484934 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a2a659-51cc-462f-bda1-0e341f39357f None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:22.519020 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c617f-1882-4d0e-b343-dc2a64df6718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:22.519626 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c617f-1882-4d0e-b343-dc2a64df6718 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:22.519901 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c617f-1882-4d0e-b343-dc2a64df6718 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:22.520171 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c617f-1882-4d0e-b343-dc2a64df6718 None admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:22.520854 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c617f-1882-4d0e-b343-dc2a64df6718 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:22.521158 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c617f-1882-4d0e-b343-dc2a64df6718 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:22.522583 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c617f-1882-4d0e-b343-dc2a64df6718 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:22.537046 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 539/1076] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 14:14:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:22.543644 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c3433-8301-436f-a603-fc5ae33f77ed None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:22.585544 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3501c4c-1097-4910-a56a-a82e077cb53b None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:22.585755 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3501c4c-1097-4910-a56a-a82e077cb53b None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:22.586031 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3501c4c-1097-4910-a56a-a82e077cb53b None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:22.587889 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 540/1077] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:14:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:14:22.592724 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030807b-28b7-4c18-be17-a9dfcbc1b959 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:22.593004 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030807b-28b7-4c18-be17-a9dfcbc1b959 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:22.593276 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030807b-28b7-4c18-be17-a9dfcbc1b959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:22.611395 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-8030807b-28b7-4c18-be17-a9dfcbc1b959 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:14:22.612566 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-8030807b-28b7-4c18-be17-a9dfcbc1b959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:14:22.645903 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c3433-8301-436f-a603-fc5ae33f77ed None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:22.653866 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:22.672028 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:22.679353 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 541/1078] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:14:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:14:22.681244 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c2ac4-d3b1-4ecd-8afd-06b6d6b8ad81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:22.681899 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c2ac4-d3b1-4ecd-8afd-06b6d6b8ad81 None admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:22.682175 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c2ac4-d3b1-4ecd-8afd-06b6d6b8ad81 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:22.682444 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c2ac4-d3b1-4ecd-8afd-06b6d6b8ad81 None admin] PATH_INFO: `/v3/projects/36a3ae7bab25471fb989136150b6438c/users/79eaaae97aac4282a613939e5b768376/roles/30791f3f2a7841d2a80aab92cf4f23f5` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:22.684558 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291c2ac4-d3b1-4ecd-8afd-06b6d6b8ad81 None admin] RBAC: Authorizing `identity:create_grant(project_id=36a3ae7bab25471fb989136150b6438c, user_id=79eaaae97aac4282a613939e5b768376, role_id=30791f3f2a7841d2a80aab92cf4f23f5)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:22.685987 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291c2ac4-d3b1-4ecd-8afd-06b6d6b8ad81 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:22.698745 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 538/1079] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 14:14:22 2022] PUT /identity/v3/projects/36a3ae7bab25471fb989136150b6438c/users/79eaaae97aac4282a613939e5b768376/roles/30791f3f2a7841d2a80aab92cf4f23f5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:14:22.948601 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8030807b-28b7-4c18-be17-a9dfcbc1b959 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:23.045308 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8030807b-28b7-4c18-be17-a9dfcbc1b959 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:23.078909 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8d2c32-1dc4-4792-af81-f5740f2b1004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:23.079523 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d2c32-1dc4-4792-af81-f5740f2b1004 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:23.079795 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d2c32-1dc4-4792-af81-f5740f2b1004 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:23.080086 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d2c32-1dc4-4792-af81-f5740f2b1004 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:23.090251 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8d2c32-1dc4-4792-af81-f5740f2b1004 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:14:23.095042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-3b8d2c32-1dc4-4792-af81-f5740f2b1004 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:14:23.095042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:23.095042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:23.095042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:23.095042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:23.095042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:23.095042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:23.095042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:23.095042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:23.095042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:23.095042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:23.095042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:23.095042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:14:23.095042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:14:23.095042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:14:23.095042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:14:23.096222 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:14:23.096222 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:14:23.096222 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:14:23.096222 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:14:23.096222 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:23.096222 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:23.096222 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:14:23.096222 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:14:23.096222 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:14:23.096222 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:14:23.096222 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:14:23.096222 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:14:23.096222 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:14:23.096222 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:14:23.096222 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:14:23.096222 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:14:23.097114 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:14:23.097114 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:14:23.097114 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:14:23.097114 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:14:23.097114 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:14:23.097114 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:14:23.097114 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:14:23.097114 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:14:23.097114 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:14:23.097114 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:14:23.097114 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:14:23.097114 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:14:23.097114 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 542/1080] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 14:14:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:14:23.102250 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c2ac4-d3b1-4ecd-8afd-06b6d6b8ad81 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:23.217922 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c2ac4-d3b1-4ecd-8afd-06b6d6b8ad81 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:23.250233 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b5134-0c6e-4f5d-9ac3-28bee453603b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:23.250858 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b5134-0c6e-4f5d-9ac3-28bee453603b None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:23.251138 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b5134-0c6e-4f5d-9ac3-28bee453603b None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:23.251404 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b5134-0c6e-4f5d-9ac3-28bee453603b None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:23.252059 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8b5134-0c6e-4f5d-9ac3-28bee453603b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:23.252352 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8b5134-0c6e-4f5d-9ac3-28bee453603b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:23.253698 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8b5134-0c6e-4f5d-9ac3-28bee453603b None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:23.262565 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 539/1081] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:14:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:23.268479 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8d2c32-1dc4-4792-af81-f5740f2b1004 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:23.303002 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:23.365532 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8d2c32-1dc4-4792-af81-f5740f2b1004 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:23.398950 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f08f09-6601-4f6f-84f4-a24ff7f8ec0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:23.399573 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f08f09-6601-4f6f-84f4-a24ff7f8ec0d None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:23.399846 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f08f09-6601-4f6f-84f4-a24ff7f8ec0d None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:23.400111 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f08f09-6601-4f6f-84f4-a24ff7f8ec0d None admin] PATH_INFO: `/v3/users/3dd3045794324575872b0719cd534e3b` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:23.401209 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f08f09-6601-4f6f-84f4-a24ff7f8ec0d None admin] RBAC: Authorizing `identity:get_user(user_id=3dd3045794324575872b0719cd534e3b)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:23.402636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f08f09-6601-4f6f-84f4-a24ff7f8ec0d None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:23.404253 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 543/1082] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:14:23 2022] GET /identity/v3/users/3dd3045794324575872b0719cd534e3b => generated 285 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:23.410525 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b5134-0c6e-4f5d-9ac3-28bee453603b None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:23.457957 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:23.515340 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b5134-0c6e-4f5d-9ac3-28bee453603b None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:23.548347 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d595de-3e92-4a93-b657-a360c2f4b92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:23.548983 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d595de-3e92-4a93-b657-a360c2f4b92d None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:23.549256 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d595de-3e92-4a93-b657-a360c2f4b92d None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:23.549523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d595de-3e92-4a93-b657-a360c2f4b92d None admin] PATH_INFO: `/v3/projects/c7c889300bce4c7e9bddec70ac0d5aa1` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:23.550714 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d595de-3e92-4a93-b657-a360c2f4b92d None admin] RBAC: Authorizing `identity:get_project(project_id=c7c889300bce4c7e9bddec70ac0d5aa1)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:23.552185 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d595de-3e92-4a93-b657-a360c2f4b92d None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:23.553895 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 540/1083] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:14:23 2022] GET /identity/v3/projects/c7c889300bce4c7e9bddec70ac0d5aa1 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:23.560316 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f08f09-6601-4f6f-84f4-a24ff7f8ec0d None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:23.656253 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f08f09-6601-4f6f-84f4-a24ff7f8ec0d None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:23.658752 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:23.689064 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04044be1-7dba-4e1e-8854-670c6c25efc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:23.689706 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04044be1-7dba-4e1e-8854-670c6c25efc6 None admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:23.689979 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04044be1-7dba-4e1e-8854-670c6c25efc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:23.690246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04044be1-7dba-4e1e-8854-670c6c25efc6 None admin] PATH_INFO: `/v3/projects/c7c889300bce4c7e9bddec70ac0d5aa1/users/3dd3045794324575872b0719cd534e3b/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:23.692118 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04044be1-7dba-4e1e-8854-670c6c25efc6 None admin] RBAC: Authorizing `identity:create_grant(project_id=c7c889300bce4c7e9bddec70ac0d5aa1, user_id=3dd3045794324575872b0719cd534e3b, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:23.693531 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04044be1-7dba-4e1e-8854-670c6c25efc6 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:23.706176 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 544/1084] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 14:14:23 2022] PUT /identity/v3/projects/c7c889300bce4c7e9bddec70ac0d5aa1/users/3dd3045794324575872b0719cd534e3b/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:14:24.819267 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c377e3b-1187-4f16-bbdc-9aa58fbae301 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:24.819802 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c377e3b-1187-4f16-bbdc-9aa58fbae301 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:24.819802 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c377e3b-1187-4f16-bbdc-9aa58fbae301 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:24.821164 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 541/1085] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:14:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:14:24.826259 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5600f-3f3a-4125-b6cf-5e1109d39fe3 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:24.826688 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5600f-3f3a-4125-b6cf-5e1109d39fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:24.826782 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5600f-3f3a-4125-b6cf-5e1109d39fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:24.845042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-0ab5600f-3f3a-4125-b6cf-5e1109d39fe3 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:14:24.846207 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-0ab5600f-3f3a-4125-b6cf-5e1109d39fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:14:24.911977 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 545/1086] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:14:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:14:25.177801 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c377e3b-1187-4f16-bbdc-9aa58fbae301 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:25.275837 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c377e3b-1187-4f16-bbdc-9aa58fbae301 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:25.292527 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:25.309580 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4473541-3c45-4daf-aea1-4aae10b8c1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:25.310201 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4473541-3c45-4daf-aea1-4aae10b8c1fe None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:25.310474 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4473541-3c45-4daf-aea1-4aae10b8c1fe None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:25.310741 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4473541-3c45-4daf-aea1-4aae10b8c1fe None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:25.320480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4473541-3c45-4daf-aea1-4aae10b8c1fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:14:25.325166 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-b4473541-3c45-4daf-aea1-4aae10b8c1fe None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 14:14:25.325166 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:25.325166 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:25.325166 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:25.325166 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:25.325166 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:25.325166 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:25.325166 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:25.325166 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:25.325166 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:25.325166 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:25.325166 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:25.325166 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:14:25.325166 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:14:25.325166 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:14:25.325166 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:14:25.326089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:14:25.326089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:14:25.326089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:14:25.326089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:14:25.326089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:25.326089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:25.326089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:14:25.326089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:14:25.326089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:14:25.326089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:14:25.326089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:14:25.326089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:14:25.326089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:14:25.326089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:14:25.326089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:14:25.326089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:14:25.327134 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:14:25.327134 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:14:25.327134 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:14:25.327134 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:14:25.327134 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:14:25.327134 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:14:25.327134 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:14:25.327134 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:14:25.327134 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:14:25.327134 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:14:25.327134 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 14:14:25.327134 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:25.327134 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 542/1087] 10.222.0.52 () {56 vars in 1120 bytes} [Sat Sep 24 14:14:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:14:25.332179 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5600f-3f3a-4125-b6cf-5e1109d39fe3 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:25.423429 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:25.431583 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5600f-3f3a-4125-b6cf-5e1109d39fe3 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:25.465426 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df7ad0-5b2e-44f0-82b3-3413d08f3169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:25.466125 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df7ad0-5b2e-44f0-82b3-3413d08f3169 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:25.466466 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df7ad0-5b2e-44f0-82b3-3413d08f3169 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:25.466799 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df7ad0-5b2e-44f0-82b3-3413d08f3169 None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:25.467492 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78df7ad0-5b2e-44f0-82b3-3413d08f3169 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:25.467790 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78df7ad0-5b2e-44f0-82b3-3413d08f3169 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:25.469158 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78df7ad0-5b2e-44f0-82b3-3413d08f3169 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:25.477770 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 546/1088] 10.222.0.52 () {56 vars in 1115 bytes} [Sat Sep 24 14:14:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:25.483516 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4473541-3c45-4daf-aea1-4aae10b8c1fe None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:25.593035 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4473541-3c45-4daf-aea1-4aae10b8c1fe None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:25.625568 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e25091b-75e9-418a-bf3f-6904765d9741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:25.626185 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25091b-75e9-418a-bf3f-6904765d9741 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:25.626457 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25091b-75e9-418a-bf3f-6904765d9741 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:25.626720 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25091b-75e9-418a-bf3f-6904765d9741 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:25.636283 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e25091b-75e9-418a-bf3f-6904765d9741 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:25.637703 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e25091b-75e9-418a-bf3f-6904765d9741 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:25.652039 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-0e25091b-75e9-418a-bf3f-6904765d9741 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:14:25.652039 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:25.652039 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:25.652039 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:25.652039 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:25.652039 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:25.652039 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:25.652039 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:25.652039 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:25.652039 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:25.652039 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:25.652039 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:25.652039 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:14:25.652039 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:14:25.652039 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:14:25.652039 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:14:25.653165 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:25.653165 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:25.653165 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:14:25.653165 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:14:25.653165 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:14:25.653165 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:14:25.653165 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:14:25.653165 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:14:25.653165 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:14:25.653165 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:14:25.653165 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:14:25.653165 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:14:25.653165 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:14:25.653165 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:14:25.653165 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:14:25.653165 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:14:25.653165 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:14:25.654339 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:14:25.654339 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:14:25.654339 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:25.655151 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 543/1089] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:14:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:14:25.659031 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df7ad0-5b2e-44f0-82b3-3413d08f3169 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:25.760751 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df7ad0-5b2e-44f0-82b3-3413d08f3169 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:25.795472 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d5e377-843c-4080-a90c-7a393b55ee3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:25.796073 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5e377-843c-4080-a90c-7a393b55ee3e None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:25.796346 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5e377-843c-4080-a90c-7a393b55ee3e None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:25.796612 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5e377-843c-4080-a90c-7a393b55ee3e None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:25.797242 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d5e377-843c-4080-a90c-7a393b55ee3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:25.797537 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d5e377-843c-4080-a90c-7a393b55ee3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:25.798932 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d5e377-843c-4080-a90c-7a393b55ee3e None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:25.810659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 547/1090] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:14:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:25.816250 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e25091b-75e9-418a-bf3f-6904765d9741 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:25.843755 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a306d-d1c3-4592-b74e-cd52fc66f60f None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:25.844203 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a306d-d1c3-4592-b74e-cd52fc66f60f None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:25.844298 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a306d-d1c3-4592-b74e-cd52fc66f60f None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:25.845595 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 548/1091] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:14:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:14:25.846949 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:25.850511 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d110d90-486c-456e-8548-e0f2d7636d37 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:25.850786 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d110d90-486c-456e-8548-e0f2d7636d37 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:25.851078 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d110d90-486c-456e-8548-e0f2d7636d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:25.868313 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-5d110d90-486c-456e-8548-e0f2d7636d37 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:14:25.869494 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-5d110d90-486c-456e-8548-e0f2d7636d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:14:25.881137 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:25.920819 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e25091b-75e9-418a-bf3f-6904765d9741 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:25.938652 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 549/1092] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:14:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:14:25.954035 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cde3f5-e6bc-4dfe-8e70-505539bba7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:25.954659 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cde3f5-e6bc-4dfe-8e70-505539bba7c0 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:25.954980 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cde3f5-e6bc-4dfe-8e70-505539bba7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:25.955256 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cde3f5-e6bc-4dfe-8e70-505539bba7c0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:25.963572 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cde3f5-e6bc-4dfe-8e70-505539bba7c0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:25.965003 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cde3f5-e6bc-4dfe-8e70-505539bba7c0 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:25.975916 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-e1cde3f5-e6bc-4dfe-8e70-505539bba7c0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 14:14:25.975916 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:25.975916 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:25.975916 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:25.975916 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:25.975916 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:25.975916 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:25.975916 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:25.975916 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:25.975916 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:25.975916 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:25.975916 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:25.975916 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:14:25.975916 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:14:25.975916 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:14:25.975916 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:14:25.981638 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:25.981638 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:25.981638 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:14:25.981638 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:14:25.981638 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:14:25.981638 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:14:25.981638 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:14:25.981638 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:14:25.981638 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:14:25.981638 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:14:25.981638 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:14:25.981638 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:14:25.981638 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:14:25.981638 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:14:25.981638 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:14:25.981638 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:14:25.981638 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:14:25.981638 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:14:25.982641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:14:25.982641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:14:25.982641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:14:25.982641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:14:25.982641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:14:25.982641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:14:25.982641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:14:25.982641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:14:25.982641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 14:14:25.982641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:25.982641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 544/1093] 10.222.0.52 () {56 vars in 1093 bytes} [Sat Sep 24 14:14:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:14:25.983375 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d110d90-486c-456e-8548-e0f2d7636d37 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:26.080237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d110d90-486c-456e-8548-e0f2d7636d37 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:26.114090 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110a5d07-abf2-41a5-828a-c041dc7dc20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:26.114686 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a5d07-abf2-41a5-828a-c041dc7dc20c None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:26.114993 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a5d07-abf2-41a5-828a-c041dc7dc20c None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:26.115267 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a5d07-abf2-41a5-828a-c041dc7dc20c None admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:26.115993 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110a5d07-abf2-41a5-828a-c041dc7dc20c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:26.116299 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110a5d07-abf2-41a5-828a-c041dc7dc20c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:26.117762 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110a5d07-abf2-41a5-828a-c041dc7dc20c None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:26.134153 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 550/1094] 10.222.0.52 () {56 vars in 1133 bytes} [Sat Sep 24 14:14:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:14:26.139913 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cde3f5-e6bc-4dfe-8e70-505539bba7c0 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:26.211999 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110a5d07-abf2-41a5-828a-c041dc7dc20c None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:26.216442 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:26.227661 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:26.240105 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cde3f5-e6bc-4dfe-8e70-505539bba7c0 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:26.273043 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b65587-e3ff-470b-9496-f2191d1dddc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:26.273638 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b65587-e3ff-470b-9496-f2191d1dddc0 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:26.273907 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b65587-e3ff-470b-9496-f2191d1dddc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:26.274172 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b65587-e3ff-470b-9496-f2191d1dddc0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:26.311412 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110a5d07-abf2-41a5-828a-c041dc7dc20c None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:26.340390 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b65587-e3ff-470b-9496-f2191d1dddc0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:26.341762 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b65587-e3ff-470b-9496-f2191d1dddc0 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:26.345010 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f3bcf5-3d2b-4bbe-9af5-61868419d671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:26.345631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f3bcf5-3d2b-4bbe-9af5-61868419d671 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:26.345909 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f3bcf5-3d2b-4bbe-9af5-61868419d671 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:26.346179 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f3bcf5-3d2b-4bbe-9af5-61868419d671 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:26.356286 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f3bcf5-3d2b-4bbe-9af5-61868419d671 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:14:26.361030 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-83f3bcf5-3d2b-4bbe-9af5-61868419d671 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:14:26.361030 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:26.361030 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:26.361030 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:26.361030 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:26.361030 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:26.361030 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:26.361030 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:26.361030 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:26.361030 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:26.361030 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:26.361030 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:26.361030 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:14:26.361030 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:14:26.361030 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:14:26.361030 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:14:26.362148 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:14:26.362148 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:14:26.362148 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:14:26.362148 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:14:26.362148 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:26.362148 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:26.362148 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:14:26.362148 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:14:26.362148 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:14:26.362148 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:14:26.362148 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:14:26.362148 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:14:26.362148 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:14:26.362148 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:14:26.362148 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:14:26.362148 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:14:26.362997 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:14:26.362997 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:14:26.362997 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:14:26.362997 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:14:26.362997 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:14:26.362997 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:14:26.362997 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:14:26.362997 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:14:26.362997 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:14:26.362997 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:14:26.362997 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:14:26.362997 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:14:26.362997 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 551/1095] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 14:14:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:14:26.368498 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f3bcf5-3d2b-4bbe-9af5-61868419d671 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:26.429737 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:26.430770 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:26.443398 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 545/1096] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:14:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 14:14:26.449032 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b65587-e3ff-470b-9496-f2191d1dddc0 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:26.466955 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f3bcf5-3d2b-4bbe-9af5-61868419d671 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:26.500305 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133ccb58-0a6d-4523-a09e-83e046fd759f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:26.500908 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ccb58-0a6d-4523-a09e-83e046fd759f None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:26.501186 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ccb58-0a6d-4523-a09e-83e046fd759f None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:26.501462 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ccb58-0a6d-4523-a09e-83e046fd759f None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:26.502119 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133ccb58-0a6d-4523-a09e-83e046fd759f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:26.502421 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133ccb58-0a6d-4523-a09e-83e046fd759f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:26.503886 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133ccb58-0a6d-4523-a09e-83e046fd759f None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:26.511852 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 552/1097] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:14:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:26.518064 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133ccb58-0a6d-4523-a09e-83e046fd759f None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:26.545500 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b65587-e3ff-470b-9496-f2191d1dddc0 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:26.579357 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d009d-6e89-44d0-b10d-82c8afe1e1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:26.579984 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d009d-6e89-44d0-b10d-82c8afe1e1ce None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:26.580260 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d009d-6e89-44d0-b10d-82c8afe1e1ce None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:26.580530 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d009d-6e89-44d0-b10d-82c8afe1e1ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:26.590804 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495d009d-6e89-44d0-b10d-82c8afe1e1ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:26.592205 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495d009d-6e89-44d0-b10d-82c8afe1e1ce None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:26.607396 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-495d009d-6e89-44d0-b10d-82c8afe1e1ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:14:26.607396 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:26.607396 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:26.607396 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:26.607396 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:26.607396 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:26.607396 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:26.607396 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:26.607396 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:26.607396 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:26.607396 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:26.607396 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:26.607396 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:14:26.607396 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:14:26.607396 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:14:26.607396 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:14:26.608391 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:26.608391 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:26.608391 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:14:26.608391 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:14:26.608391 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:14:26.608391 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:14:26.608391 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:14:26.608391 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:14:26.608391 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:14:26.608391 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:14:26.608391 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:14:26.608391 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:14:26.608391 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:14:26.608391 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:14:26.608391 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:14:26.608391 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:14:26.608391 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:14:26.609308 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:14:26.609308 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:14:26.609308 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:26.613154 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 546/1098] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:14:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:14:26.613945 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d009d-6e89-44d0-b10d-82c8afe1e1ce None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:26.616263 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133ccb58-0a6d-4523-a09e-83e046fd759f None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:26.641841 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:26.651660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66610bf-5b72-4359-b29d-316a87e0fade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:26.652283 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66610bf-5b72-4359-b29d-316a87e0fade None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:26.652561 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66610bf-5b72-4359-b29d-316a87e0fade None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:26.652833 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66610bf-5b72-4359-b29d-316a87e0fade None admin] PATH_INFO: `/v3/users/3dd3045794324575872b0719cd534e3b` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:26.653953 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66610bf-5b72-4359-b29d-316a87e0fade None admin] RBAC: Authorizing `identity:get_user(user_id=3dd3045794324575872b0719cd534e3b)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:26.655420 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66610bf-5b72-4359-b29d-316a87e0fade None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:26.656053 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:26.657052 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 553/1099] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:14:26 2022] GET /identity/v3/users/3dd3045794324575872b0719cd534e3b => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:14:26.663205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66610bf-5b72-4359-b29d-316a87e0fade None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:26.715761 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d009d-6e89-44d0-b10d-82c8afe1e1ce None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:26.748737 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b3044f-a262-4662-8552-5e1efe00baba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:26.749341 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3044f-a262-4662-8552-5e1efe00baba None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:26.749616 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3044f-a262-4662-8552-5e1efe00baba None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:26.749888 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3044f-a262-4662-8552-5e1efe00baba None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:26.750525 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b3044f-a262-4662-8552-5e1efe00baba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:26.750854 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b3044f-a262-4662-8552-5e1efe00baba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:26.752207 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b3044f-a262-4662-8552-5e1efe00baba None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:26.763202 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66610bf-5b72-4359-b29d-316a87e0fade None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:26.763584 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 547/1100] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:14:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:14:26.769244 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b3044f-a262-4662-8552-5e1efe00baba None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:26.797968 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9e078-4f89-4548-8ead-2c97e5c3112d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:26.798571 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9e078-4f89-4548-8ead-2c97e5c3112d None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:26.798878 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9e078-4f89-4548-8ead-2c97e5c3112d None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:26.799148 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9e078-4f89-4548-8ead-2c97e5c3112d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:26.864019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:26.866222 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:26.868981 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b3044f-a262-4662-8552-5e1efe00baba None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:26.871872 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe9e078-4f89-4548-8ead-2c97e5c3112d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:26.873317 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe9e078-4f89-4548-8ead-2c97e5c3112d None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:26.902874 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77342ecd-4977-46aa-86e4-1edf11428804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:26.903509 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77342ecd-4977-46aa-86e4-1edf11428804 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:26.903794 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77342ecd-4977-46aa-86e4-1edf11428804 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:26.904065 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77342ecd-4977-46aa-86e4-1edf11428804 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:26.911479 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77342ecd-4977-46aa-86e4-1edf11428804 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:26.912915 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77342ecd-4977-46aa-86e4-1edf11428804 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:26.924787 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-77342ecd-4977-46aa-86e4-1edf11428804 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:14:26.924787 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:26.924787 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:26.924787 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:26.924787 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:26.924787 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:26.924787 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:26.924787 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:26.924787 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:26.924787 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:26.924787 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:26.924787 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:26.924787 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:14:26.924787 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:14:26.924787 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:14:26.925762 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:14:26.925762 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:26.925762 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:26.925762 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:14:26.925762 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:14:26.925762 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:14:26.925762 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:14:26.925762 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:14:26.925762 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:14:26.925762 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:14:26.925762 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:14:26.925762 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:14:26.925762 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:14:26.925762 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:14:26.925762 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:14:26.925762 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:14:26.925762 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:14:26.926681 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:14:26.926681 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:14:26.926681 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:14:26.926681 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:14:26.926681 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:14:26.926681 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:14:26.926681 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:14:26.926681 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:26.930937 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 548/1101] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:14:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:14:26.931751 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77342ecd-4977-46aa-86e4-1edf11428804 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:26.973901 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 554/1102] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:14:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 24 14:14:26.980338 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9e078-4f89-4548-8ead-2c97e5c3112d None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:27.035325 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77342ecd-4977-46aa-86e4-1edf11428804 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:27.069209 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7430063f-2ed7-4e49-8f93-5ed63e0809fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:27.069812 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430063f-2ed7-4e49-8f93-5ed63e0809fd None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:27.070086 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430063f-2ed7-4e49-8f93-5ed63e0809fd None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:27.070357 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430063f-2ed7-4e49-8f93-5ed63e0809fd None admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:27.071086 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7430063f-2ed7-4e49-8f93-5ed63e0809fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:27.071394 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7430063f-2ed7-4e49-8f93-5ed63e0809fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:27.071606 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:27.072827 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7430063f-2ed7-4e49-8f93-5ed63e0809fd None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:27.078218 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9e078-4f89-4548-8ead-2c97e5c3112d None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:27.084591 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 549/1103] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 14:14:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:14:27.091177 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7430063f-2ed7-4e49-8f93-5ed63e0809fd None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:27.113561 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:27.113859 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695af2d5-4832-4020-aa63-5324a5c92625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:27.114748 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695af2d5-4832-4020-aa63-5324a5c92625 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:27.114927 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695af2d5-4832-4020-aa63-5324a5c92625 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:27.115271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695af2d5-4832-4020-aa63-5324a5c92625 None admin] PATH_INFO: `/v3/projects/c7c889300bce4c7e9bddec70ac0d5aa1` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:27.116430 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695af2d5-4832-4020-aa63-5324a5c92625 None admin] RBAC: Authorizing `identity:get_project(project_id=c7c889300bce4c7e9bddec70ac0d5aa1)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:27.117872 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695af2d5-4832-4020-aa63-5324a5c92625 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:27.119625 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 555/1104] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:14:26 2022] GET /identity/v3/projects/c7c889300bce4c7e9bddec70ac0d5aa1 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:14:27.125713 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695af2d5-4832-4020-aa63-5324a5c92625 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:27.195128 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7430063f-2ed7-4e49-8f93-5ed63e0809fd None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:27.230375 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793f7f95-9532-40ea-9322-287a3f49c8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:27.231050 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f7f95-9532-40ea-9322-287a3f49c8fd None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:27.231331 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f7f95-9532-40ea-9322-287a3f49c8fd None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:27.231599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f7f95-9532-40ea-9322-287a3f49c8fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:27.232320 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793f7f95-9532-40ea-9322-287a3f49c8fd None admin] RBAC: Adding query filter params (role.id=30791f3f2a7841d2a80aab92cf4f23f5, scope.project.id=36a3ae7bab25471fb989136150b6438c, user.id=79eaaae97aac4282a613939e5b768376) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:27.232619 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793f7f95-9532-40ea-9322-287a3f49c8fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:27.234109 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793f7f95-9532-40ea-9322-287a3f49c8fd None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:27.239943 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 550/1105] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:14:27 2022] GET /identity/v3/role_assignments?user.id=79eaaae97aac4282a613939e5b768376&scope.project.id=36a3ae7bab25471fb989136150b6438c&role.id=30791f3f2a7841d2a80aab92cf4f23f5 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:14:27.258661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695af2d5-4832-4020-aa63-5324a5c92625 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:27.302009 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f8d4f-c68c-4486-881c-f8b13c41ac9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:27.302626 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f8d4f-c68c-4486-881c-f8b13c41ac9a None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:27.302940 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f8d4f-c68c-4486-881c-f8b13c41ac9a None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:27.303213 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f8d4f-c68c-4486-881c-f8b13c41ac9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:27.303959 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f8d4f-c68c-4486-881c-f8b13c41ac9a None admin] RBAC: Adding query filter params (role.id=d00b78c32dbf4cc7925a55bb621d8fcf, scope.project.id=c7c889300bce4c7e9bddec70ac0d5aa1, user.id=3dd3045794324575872b0719cd534e3b) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:27.304265 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f8d4f-c68c-4486-881c-f8b13c41ac9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:27.306203 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f8d4f-c68c-4486-881c-f8b13c41ac9a None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:27.315187 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 556/1106] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:14:27 2022] GET /identity/v3/role_assignments?user.id=3dd3045794324575872b0719cd534e3b&scope.project.id=c7c889300bce4c7e9bddec70ac0d5aa1&role.id=d00b78c32dbf4cc7925a55bb621d8fcf => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:14:29.552296 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8996c7-c7b3-4daf-a725-fe892b01be90 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:29.553867 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8996c7-c7b3-4daf-a725-fe892b01be90 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:29.553867 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8996c7-c7b3-4daf-a725-fe892b01be90 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:29.554162 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 551/1107] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:14:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:14:29.560028 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbd6e9-7971-4f17-a821-07676fd85539 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:29.560474 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbd6e9-7971-4f17-a821-07676fd85539 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:29.560574 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbd6e9-7971-4f17-a821-07676fd85539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:29.578696 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-d1cbd6e9-7971-4f17-a821-07676fd85539 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:14:29.579888 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-d1cbd6e9-7971-4f17-a821-07676fd85539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:14:29.599728 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:29.649623 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 557/1108] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:14:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:14:29.913673 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8996c7-c7b3-4daf-a725-fe892b01be90 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:30.004002 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:30.020953 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8996c7-c7b3-4daf-a725-fe892b01be90 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:30.053117 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9a8b6-f28b-4cab-ad57-98f12aa2729f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:30.053738 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9a8b6-f28b-4cab-ad57-98f12aa2729f None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:30.054037 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9a8b6-f28b-4cab-ad57-98f12aa2729f None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:30.054307 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9a8b6-f28b-4cab-ad57-98f12aa2729f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:30.055964 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9a8b6-f28b-4cab-ad57-98f12aa2729f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:30.057370 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9a8b6-f28b-4cab-ad57-98f12aa2729f None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:30.058938 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 552/1109] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:14:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:30.064671 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cbd6e9-7971-4f17-a821-07676fd85539 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:30.160039 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cbd6e9-7971-4f17-a821-07676fd85539 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:30.193268 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d213416-4fbb-4133-901d-a3fb02d2d1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:30.193860 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d213416-4fbb-4133-901d-a3fb02d2d1ad None admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:30.194135 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d213416-4fbb-4133-901d-a3fb02d2d1ad None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:30.194398 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d213416-4fbb-4133-901d-a3fb02d2d1ad None admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:30.195058 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d213416-4fbb-4133-901d-a3fb02d2d1ad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:30.196470 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d213416-4fbb-4133-901d-a3fb02d2d1ad None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:30.216608 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 558/1110] 10.222.0.52 () {60 vars in 1132 bytes} [Sat Sep 24 14:14:30 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:14:32.421913 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045559c-7ec0-4909-b6ef-e3c9e0f1ff5f None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:32.422477 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045559c-7ec0-4909-b6ef-e3c9e0f1ff5f None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:32.422477 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045559c-7ec0-4909-b6ef-e3c9e0f1ff5f None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:32.423815 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 553/1111] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:14:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:14:32.429085 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58648b62-f938-404b-b1b5-8ad19eaf2996 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:32.429479 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58648b62-f938-404b-b1b5-8ad19eaf2996 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:32.429665 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58648b62-f938-404b-b1b5-8ad19eaf2996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:32.447880 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-58648b62-f938-404b-b1b5-8ad19eaf2996 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:14:32.449082 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-58648b62-f938-404b-b1b5-8ad19eaf2996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:14:32.456130 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:32.514279 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 559/1112] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:14:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:14:32.781149 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0045559c-7ec0-4909-b6ef-e3c9e0f1ff5f None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:32.878805 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0045559c-7ec0-4909-b6ef-e3c9e0f1ff5f None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:32.911543 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ba6c3-193d-460c-b68d-d15b301e953d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:32.912161 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ba6c3-193d-460c-b68d-d15b301e953d None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:32.912439 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ba6c3-193d-460c-b68d-d15b301e953d None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:32.912708 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ba6c3-193d-460c-b68d-d15b301e953d None admin] PATH_INFO: `/v3/roles/25257d00ad2640c2a57d257b0279155f` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:32.914117 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427ba6c3-193d-460c-b68d-d15b301e953d None admin] RBAC: Authorizing `identity:get_role(role_id=25257d00ad2640c2a57d257b0279155f)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:32.915560 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427ba6c3-193d-460c-b68d-d15b301e953d None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:32.916887 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 554/1113] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:14:32 2022] GET /identity/v3/roles/25257d00ad2640c2a57d257b0279155f => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:32.922654 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58648b62-f938-404b-b1b5-8ad19eaf2996 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:33.021994 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58648b62-f938-404b-b1b5-8ad19eaf2996 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:33.054330 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffcf643-8b27-466c-9a4c-85c785503b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:33.055414 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffcf643-8b27-466c-9a4c-85c785503b95 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:33.055691 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffcf643-8b27-466c-9a4c-85c785503b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:33.055960 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffcf643-8b27-466c-9a4c-85c785503b95 None admin] PATH_INFO: `/v3/users/64f36f6e33ec4d0694d92a6ef3cc8653` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:33.066430 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:33.069246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffcf643-8b27-466c-9a4c-85c785503b95 None admin] RBAC: Authorizing `identity:get_user(user_id=64f36f6e33ec4d0694d92a6ef3cc8653)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:33.071679 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffcf643-8b27-466c-9a4c-85c785503b95 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:33.073296 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 560/1114] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:14:32 2022] GET /identity/v3/users/64f36f6e33ec4d0694d92a6ef3cc8653 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:33.079180 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ba6c3-193d-460c-b68d-d15b301e953d None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:33.100574 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:33.175047 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ba6c3-193d-460c-b68d-d15b301e953d None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:33.207106 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71001d3a-0715-4d36-baca-1d0dde61b36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:33.207693 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71001d3a-0715-4d36-baca-1d0dde61b36c None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:33.207976 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71001d3a-0715-4d36-baca-1d0dde61b36c None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:33.208245 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71001d3a-0715-4d36-baca-1d0dde61b36c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:33.277420 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71001d3a-0715-4d36-baca-1d0dde61b36c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:33.278904 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71001d3a-0715-4d36-baca-1d0dde61b36c None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:33.301404 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:33.378951 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 555/1115] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:14:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 14:14:33.384770 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffcf643-8b27-466c-9a4c-85c785503b95 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:33.479645 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffcf643-8b27-466c-9a4c-85c785503b95 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:33.512319 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad642b-75a0-4120-9371-39d694be15ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:33.512937 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad642b-75a0-4120-9371-39d694be15ae None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:33.513214 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad642b-75a0-4120-9371-39d694be15ae None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:33.513484 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad642b-75a0-4120-9371-39d694be15ae None admin] PATH_INFO: `/v3/projects/c7c889300bce4c7e9bddec70ac0d5aa1` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:33.514574 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dad642b-75a0-4120-9371-39d694be15ae None admin] RBAC: Authorizing `identity:get_project(project_id=c7c889300bce4c7e9bddec70ac0d5aa1)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:33.516029 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dad642b-75a0-4120-9371-39d694be15ae None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:33.517624 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 561/1116] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:14:33 2022] GET /identity/v3/projects/c7c889300bce4c7e9bddec70ac0d5aa1 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:33.524192 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71001d3a-0715-4d36-baca-1d0dde61b36c None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:33.618757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71001d3a-0715-4d36-baca-1d0dde61b36c None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:33.638164 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:33.652202 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b7e57-1017-40db-b433-9f446d7dfa3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:33.652791 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b7e57-1017-40db-b433-9f446d7dfa3e None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:33.653063 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b7e57-1017-40db-b433-9f446d7dfa3e None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:33.653334 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b7e57-1017-40db-b433-9f446d7dfa3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:33.654050 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0b7e57-1017-40db-b433-9f446d7dfa3e None admin] RBAC: Adding query filter params (role.id=25257d00ad2640c2a57d257b0279155f, scope.project.id=c7c889300bce4c7e9bddec70ac0d5aa1, user.id=64f36f6e33ec4d0694d92a6ef3cc8653) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:33.654352 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0b7e57-1017-40db-b433-9f446d7dfa3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:33.655788 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0b7e57-1017-40db-b433-9f446d7dfa3e None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:33.661190 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 556/1117] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:14:33 2022] GET /identity/v3/role_assignments?user.id=64f36f6e33ec4d0694d92a6ef3cc8653&scope.project.id=c7c889300bce4c7e9bddec70ac0d5aa1&role.id=25257d00ad2640c2a57d257b0279155f => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:35.766423 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef470f1-33e3-452a-857e-7f4adae73469 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:35.766987 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef470f1-33e3-452a-857e-7f4adae73469 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:35.766987 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef470f1-33e3-452a-857e-7f4adae73469 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:35.768315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 562/1118] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:14:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:14:35.774171 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a1b51-a257-4ad1-a5e5-173e12ee09a1 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:35.774607 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a1b51-a257-4ad1-a5e5-173e12ee09a1 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:35.774708 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a1b51-a257-4ad1-a5e5-173e12ee09a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:35.794172 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-8a0a1b51-a257-4ad1-a5e5-173e12ee09a1 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:14:35.795369 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-8a0a1b51-a257-4ad1-a5e5-173e12ee09a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:14:35.859664 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 557/1119] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:14:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:14:36.132236 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef470f1-33e3-452a-857e-7f4adae73469 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:36.198790 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:36.232060 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef470f1-33e3-452a-857e-7f4adae73469 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:36.264813 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d903f90c-02aa-4061-8f87-086374c04d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:36.265432 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903f90c-02aa-4061-8f87-086374c04d7d None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:36.265711 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903f90c-02aa-4061-8f87-086374c04d7d None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:36.265981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903f90c-02aa-4061-8f87-086374c04d7d None admin] PATH_INFO: `/v3/roles/25257d00ad2640c2a57d257b0279155f` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:36.267376 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d903f90c-02aa-4061-8f87-086374c04d7d None admin] RBAC: Authorizing `identity:get_role(role_id=25257d00ad2640c2a57d257b0279155f)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:36.268740 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d903f90c-02aa-4061-8f87-086374c04d7d None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:36.270026 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 563/1120] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:14:36 2022] GET /identity/v3/roles/25257d00ad2640c2a57d257b0279155f => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:36.276318 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a1b51-a257-4ad1-a5e5-173e12ee09a1 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:36.370656 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a1b51-a257-4ad1-a5e5-173e12ee09a1 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:36.384450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:36.404263 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83b81dd-c8ca-4f63-bce4-ba276b64712a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:36.404882 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b81dd-c8ca-4f63-bce4-ba276b64712a None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:36.405162 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b81dd-c8ca-4f63-bce4-ba276b64712a None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:36.405434 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b81dd-c8ca-4f63-bce4-ba276b64712a None admin] PATH_INFO: `/v3/users/64f36f6e33ec4d0694d92a6ef3cc8653` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:36.406576 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83b81dd-c8ca-4f63-bce4-ba276b64712a None admin] RBAC: Authorizing `identity:get_user(user_id=64f36f6e33ec4d0694d92a6ef3cc8653)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:36.408021 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83b81dd-c8ca-4f63-bce4-ba276b64712a None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:36.409822 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 558/1121] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:14:36 2022] GET /identity/v3/users/64f36f6e33ec4d0694d92a6ef3cc8653 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:36.415353 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d903f90c-02aa-4061-8f87-086374c04d7d None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:36.511994 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d903f90c-02aa-4061-8f87-086374c04d7d None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:36.545233 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea69b813-5cac-4100-b702-8ada77226383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:36.545837 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69b813-5cac-4100-b702-8ada77226383 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:36.546113 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69b813-5cac-4100-b702-8ada77226383 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:36.546380 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69b813-5cac-4100-b702-8ada77226383 None admin] PATH_INFO: `/v3/projects/c7c889300bce4c7e9bddec70ac0d5aa1` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:36.547436 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea69b813-5cac-4100-b702-8ada77226383 None admin] RBAC: Authorizing `identity:get_project(project_id=c7c889300bce4c7e9bddec70ac0d5aa1)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:36.548844 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea69b813-5cac-4100-b702-8ada77226383 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:36.550430 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 564/1122] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:14:36 2022] GET /identity/v3/projects/c7c889300bce4c7e9bddec70ac0d5aa1 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:36.556031 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83b81dd-c8ca-4f63-bce4-ba276b64712a None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:36.652342 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83b81dd-c8ca-4f63-bce4-ba276b64712a None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:36.684923 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e790c24-2102-4239-aea6-9565c2b0ad03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:36.685561 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e790c24-2102-4239-aea6-9565c2b0ad03 None admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:36.685838 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e790c24-2102-4239-aea6-9565c2b0ad03 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:36.686111 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e790c24-2102-4239-aea6-9565c2b0ad03 None admin] PATH_INFO: `/v3/projects/c7c889300bce4c7e9bddec70ac0d5aa1/users/64f36f6e33ec4d0694d92a6ef3cc8653/roles/25257d00ad2640c2a57d257b0279155f` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:36.688238 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e790c24-2102-4239-aea6-9565c2b0ad03 None admin] RBAC: Authorizing `identity:create_grant(project_id=c7c889300bce4c7e9bddec70ac0d5aa1, user_id=64f36f6e33ec4d0694d92a6ef3cc8653, role_id=25257d00ad2640c2a57d257b0279155f)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:36.689650 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e790c24-2102-4239-aea6-9565c2b0ad03 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:36.701810 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 559/1123] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 14:14:36 2022] PUT /identity/v3/projects/c7c889300bce4c7e9bddec70ac0d5aa1/users/64f36f6e33ec4d0694d92a6ef3cc8653/roles/25257d00ad2640c2a57d257b0279155f => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:14:38.806222 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890d861-5e8c-4e33-a0b6-d8e96d465e33 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:38.806753 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890d861-5e8c-4e33-a0b6-d8e96d465e33 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:38.806753 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890d861-5e8c-4e33-a0b6-d8e96d465e33 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:38.808125 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 565/1124] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:14:38.813346 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901273f-af89-4ace-981e-b033488cccb6 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:38.813735 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901273f-af89-4ace-981e-b033488cccb6 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:38.813888 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901273f-af89-4ace-981e-b033488cccb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:38.832184 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-8901273f-af89-4ace-981e-b033488cccb6 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:14:38.833768 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-8901273f-af89-4ace-981e-b033488cccb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:14:38.866276 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:38.897950 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 560/1125] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:14:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:14:39.167799 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890d861-5e8c-4e33-a0b6-d8e96d465e33 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:39.171615 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:39.276869 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890d861-5e8c-4e33-a0b6-d8e96d465e33 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:39.310013 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76920731-b510-40c8-9318-1c4657732054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:39.310634 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76920731-b510-40c8-9318-1c4657732054 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:39.310927 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76920731-b510-40c8-9318-1c4657732054 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:39.311195 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76920731-b510-40c8-9318-1c4657732054 None admin] PATH_INFO: `/v3/roles/25257d00ad2640c2a57d257b0279155f` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:39.312555 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76920731-b510-40c8-9318-1c4657732054 None admin] RBAC: Authorizing `identity:get_role(role_id=25257d00ad2640c2a57d257b0279155f)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:39.313912 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76920731-b510-40c8-9318-1c4657732054 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:39.315264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 566/1126] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:14:39 2022] GET /identity/v3/roles/25257d00ad2640c2a57d257b0279155f => generated 223 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:39.321675 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8901273f-af89-4ace-981e-b033488cccb6 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:39.417738 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8901273f-af89-4ace-981e-b033488cccb6 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:39.450519 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257f9dbc-2ef3-4a97-b721-2ff552d1fe75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:39.451224 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f9dbc-2ef3-4a97-b721-2ff552d1fe75 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:39.451507 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f9dbc-2ef3-4a97-b721-2ff552d1fe75 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:39.451778 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f9dbc-2ef3-4a97-b721-2ff552d1fe75 None admin] PATH_INFO: `/v3/users/64f36f6e33ec4d0694d92a6ef3cc8653` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:39.452889 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257f9dbc-2ef3-4a97-b721-2ff552d1fe75 None admin] RBAC: Authorizing `identity:get_user(user_id=64f36f6e33ec4d0694d92a6ef3cc8653)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:39.454300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257f9dbc-2ef3-4a97-b721-2ff552d1fe75 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:39.455872 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 561/1127] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:14:39 2022] GET /identity/v3/users/64f36f6e33ec4d0694d92a6ef3cc8653 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:39.461517 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76920731-b510-40c8-9318-1c4657732054 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:39.528721 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:39.557916 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76920731-b510-40c8-9318-1c4657732054 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:39.591790 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a22bdc1-3ee3-41c6-a603-70a467106b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:39.592389 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22bdc1-3ee3-41c6-a603-70a467106b76 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:39.592662 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22bdc1-3ee3-41c6-a603-70a467106b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:39.592925 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22bdc1-3ee3-41c6-a603-70a467106b76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:39.657577 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a22bdc1-3ee3-41c6-a603-70a467106b76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:39.659310 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a22bdc1-3ee3-41c6-a603-70a467106b76 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:39.723996 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:39.753660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 567/1128] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:14:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 14:14:39.760285 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257f9dbc-2ef3-4a97-b721-2ff552d1fe75 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:39.801017 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:39.859568 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257f9dbc-2ef3-4a97-b721-2ff552d1fe75 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:39.892288 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f80af2-c90f-4f64-bc6b-97fdff158e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:39.892901 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f80af2-c90f-4f64-bc6b-97fdff158e2f None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:39.893174 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f80af2-c90f-4f64-bc6b-97fdff158e2f None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:39.893439 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f80af2-c90f-4f64-bc6b-97fdff158e2f None admin] PATH_INFO: `/v3/projects/c7c889300bce4c7e9bddec70ac0d5aa1` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:39.894523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f80af2-c90f-4f64-bc6b-97fdff158e2f None admin] RBAC: Authorizing `identity:get_project(project_id=c7c889300bce4c7e9bddec70ac0d5aa1)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:39.895962 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f80af2-c90f-4f64-bc6b-97fdff158e2f None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:39.897556 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 562/1129] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:14:39 2022] GET /identity/v3/projects/c7c889300bce4c7e9bddec70ac0d5aa1 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:14:39.903733 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a22bdc1-3ee3-41c6-a603-70a467106b76 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:39.998797 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a22bdc1-3ee3-41c6-a603-70a467106b76 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:40.030574 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6edc0-fb59-4ea9-99ae-02677faa1e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:40.031193 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6edc0-fb59-4ea9-99ae-02677faa1e99 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:40.031468 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6edc0-fb59-4ea9-99ae-02677faa1e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:40.031734 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6edc0-fb59-4ea9-99ae-02677faa1e99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:40.032445 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d6edc0-fb59-4ea9-99ae-02677faa1e99 None admin] RBAC: Adding query filter params (role.id=25257d00ad2640c2a57d257b0279155f, scope.project.id=c7c889300bce4c7e9bddec70ac0d5aa1, user.id=64f36f6e33ec4d0694d92a6ef3cc8653) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:40.032746 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d6edc0-fb59-4ea9-99ae-02677faa1e99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:40.034416 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d6edc0-fb59-4ea9-99ae-02677faa1e99 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:40.040478 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 568/1130] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:14:39 2022] GET /identity/v3/role_assignments?user.id=64f36f6e33ec4d0694d92a6ef3cc8653&scope.project.id=c7c889300bce4c7e9bddec70ac0d5aa1&role.id=25257d00ad2640c2a57d257b0279155f => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:42.189710 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5dde24-c2ac-48fd-96d3-5504888e5c72 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:42.190219 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5dde24-c2ac-48fd-96d3-5504888e5c72 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:42.190329 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5dde24-c2ac-48fd-96d3-5504888e5c72 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:42.191620 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 563/1131] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:14:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:14:42.196944 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0c35e9-83e5-4ac4-874c-7489cc55c5e4 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:42.197310 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0c35e9-83e5-4ac4-874c-7489cc55c5e4 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:42.197489 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0c35e9-83e5-4ac4-874c-7489cc55c5e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:42.215306 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-dd0c35e9-83e5-4ac4-874c-7489cc55c5e4 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:14:42.216490 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-dd0c35e9-83e5-4ac4-874c-7489cc55c5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:14:42.245217 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:42.284225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 569/1132] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:14:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:14:42.551359 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5dde24-c2ac-48fd-96d3-5504888e5c72 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:42.647964 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5dde24-c2ac-48fd-96d3-5504888e5c72 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:42.656765 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:42.681016 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bd8847-2dc5-444f-870d-26ee3d1acf0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:42.681618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd8847-2dc5-444f-870d-26ee3d1acf0f None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:42.681891 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd8847-2dc5-444f-870d-26ee3d1acf0f None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:42.682158 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd8847-2dc5-444f-870d-26ee3d1acf0f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:42.683267 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bd8847-2dc5-444f-870d-26ee3d1acf0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:42.684670 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bd8847-2dc5-444f-870d-26ee3d1acf0f None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:42.686174 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 564/1133] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:14:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:42.691831 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0c35e9-83e5-4ac4-874c-7489cc55c5e4 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:42.788134 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0c35e9-83e5-4ac4-874c-7489cc55c5e4 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:42.822051 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73dbb01-6db7-4ec3-9efb-b45a53fb3a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:42.822594 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73dbb01-6db7-4ec3-9efb-b45a53fb3a49 None admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:42.822908 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73dbb01-6db7-4ec3-9efb-b45a53fb3a49 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:42.823183 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73dbb01-6db7-4ec3-9efb-b45a53fb3a49 None admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:42.823826 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73dbb01-6db7-4ec3-9efb-b45a53fb3a49 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:42.825257 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73dbb01-6db7-4ec3-9efb-b45a53fb3a49 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:42.853189 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 570/1134] 10.222.0.52 () {60 vars in 1141 bytes} [Sat Sep 24 14:14:42 2022] POST /identity/v3/projects => generated 311 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:14:45.022455 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07a77b-8dbe-4f6e-abea-95edbe01eea9 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:45.023598 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07a77b-8dbe-4f6e-abea-95edbe01eea9 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:45.023598 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07a77b-8dbe-4f6e-abea-95edbe01eea9 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:45.024864 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 565/1135] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:14:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:14:45.033086 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51d26a-1b60-494f-a5ca-7cd850f72e6b None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:45.033810 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51d26a-1b60-494f-a5ca-7cd850f72e6b None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:45.034125 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51d26a-1b60-494f-a5ca-7cd850f72e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:45.054322 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-5c51d26a-1b60-494f-a5ca-7cd850f72e6b None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:14:45.055550 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-5c51d26a-1b60-494f-a5ca-7cd850f72e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:14:45.068487 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:45.122155 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 571/1136] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:14:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:14:45.388438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e07a77b-8dbe-4f6e-abea-95edbe01eea9 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:45.486344 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e07a77b-8dbe-4f6e-abea-95edbe01eea9 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:45.518992 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107796f0-2da2-4020-8512-d9f897d0a71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:45.519717 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107796f0-2da2-4020-8512-d9f897d0a71c None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:45.520018 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107796f0-2da2-4020-8512-d9f897d0a71c None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:45.520331 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107796f0-2da2-4020-8512-d9f897d0a71c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:45.521497 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107796f0-2da2-4020-8512-d9f897d0a71c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:45.523500 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107796f0-2da2-4020-8512-d9f897d0a71c None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:45.525049 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 566/1137] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:14:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:45.530723 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c51d26a-1b60-494f-a5ca-7cd850f72e6b None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:45.626572 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c51d26a-1b60-494f-a5ca-7cd850f72e6b None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:45.659764 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d274b-d672-4c0b-b826-04045b223147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:45.660296 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d274b-d672-4c0b-b826-04045b223147 None admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:45.660574 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d274b-d672-4c0b-b826-04045b223147 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:45.660842 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d274b-d672-4c0b-b826-04045b223147 None admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:45.661489 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d274b-d672-4c0b-b826-04045b223147 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:45.662943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d274b-d672-4c0b-b826-04045b223147 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:45.686807 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 572/1138] 10.222.0.52 () {60 vars in 1132 bytes} [Sat Sep 24 14:14:45 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:14:47.870941 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d824d-0157-4dc3-acc1-7856f0efed3c None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:47.871363 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d824d-0157-4dc3-acc1-7856f0efed3c None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:47.871484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d824d-0157-4dc3-acc1-7856f0efed3c None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:47.872826 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 567/1139] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:14:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:14:47.878172 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb735b-a57f-424f-9921-998888dead56 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:47.878597 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb735b-a57f-424f-9921-998888dead56 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:47.878695 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb735b-a57f-424f-9921-998888dead56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:47.885799 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:47.897754 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-f7eb735b-a57f-424f-9921-998888dead56 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:14:47.899478 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-f7eb735b-a57f-424f-9921-998888dead56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:14:47.963300 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 573/1140] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:14:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:14:48.233235 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d824d-0157-4dc3-acc1-7856f0efed3c None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:48.278597 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:48.331122 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d824d-0157-4dc3-acc1-7856f0efed3c None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:48.364300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc5f49-4598-45fe-b2da-5f395e8e928f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:48.364922 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc5f49-4598-45fe-b2da-5f395e8e928f None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:48.365196 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc5f49-4598-45fe-b2da-5f395e8e928f None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:48.365488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc5f49-4598-45fe-b2da-5f395e8e928f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:48.375825 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fc5f49-4598-45fe-b2da-5f395e8e928f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:14:48.380524 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-f9fc5f49-4598-45fe-b2da-5f395e8e928f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:14:48.380524 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:48.380524 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:48.380524 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:48.380524 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:48.380524 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:48.380524 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:48.380524 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:48.380524 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:48.380524 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:48.380524 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:48.380524 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:48.380524 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:14:48.380524 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:14:48.380524 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:14:48.380524 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:14:48.381732 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:14:48.381732 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:14:48.381732 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:14:48.381732 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:14:48.381732 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:48.381732 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:48.381732 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:14:48.381732 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:14:48.381732 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:14:48.381732 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:14:48.381732 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:14:48.381732 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:14:48.381732 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:14:48.381732 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:14:48.381732 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:14:48.381732 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:14:48.382849 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:14:48.382849 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:14:48.382849 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:14:48.382849 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:14:48.382849 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:14:48.382849 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:14:48.382849 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:14:48.382849 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:14:48.382849 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:14:48.382849 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:14:48.382849 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:14:48.382849 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:48.382849 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 568/1141] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 14:14:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:14:48.387778 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb735b-a57f-424f-9921-998888dead56 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:48.483185 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb735b-a57f-424f-9921-998888dead56 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:48.516531 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ec68c-58c6-485f-879b-c3367ee26b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:48.517208 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ec68c-58c6-485f-879b-c3367ee26b96 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:48.517488 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ec68c-58c6-485f-879b-c3367ee26b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:48.517765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ec68c-58c6-485f-879b-c3367ee26b96 None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:48.518419 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ec68c-58c6-485f-879b-c3367ee26b96 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:48.518766 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ec68c-58c6-485f-879b-c3367ee26b96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:48.520123 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ec68c-58c6-485f-879b-c3367ee26b96 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:48.521521 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:48.529504 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 574/1142] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:14:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:48.535693 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc5f49-4598-45fe-b2da-5f395e8e928f None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:48.632589 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc5f49-4598-45fe-b2da-5f395e8e928f None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:48.634878 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:48.666824 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7634c7d8-04af-413f-97e9-e75104b56e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:48.667463 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7634c7d8-04af-413f-97e9-e75104b56e17 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:48.667740 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7634c7d8-04af-413f-97e9-e75104b56e17 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:48.668012 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7634c7d8-04af-413f-97e9-e75104b56e17 None admin] PATH_INFO: `/v3/users/735ad65a54eb4670b9f0a6a011dc2e78` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:48.680495 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7634c7d8-04af-413f-97e9-e75104b56e17 None admin] RBAC: Authorizing `identity:get_user(user_id=735ad65a54eb4670b9f0a6a011dc2e78)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:48.681918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7634c7d8-04af-413f-97e9-e75104b56e17 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:48.683531 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 569/1143] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:14:48 2022] GET /identity/v3/users/735ad65a54eb4670b9f0a6a011dc2e78 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:48.689245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ec68c-58c6-485f-879b-c3367ee26b96 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:48.787893 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ec68c-58c6-485f-879b-c3367ee26b96 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:48.820536 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15995fd2-52d2-41cb-b827-50ba845b5b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:48.821142 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15995fd2-52d2-41cb-b827-50ba845b5b55 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:48.821418 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15995fd2-52d2-41cb-b827-50ba845b5b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:48.821687 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15995fd2-52d2-41cb-b827-50ba845b5b55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:48.881712 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:48.888662 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15995fd2-52d2-41cb-b827-50ba845b5b55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:48.890113 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15995fd2-52d2-41cb-b827-50ba845b5b55 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:48.985596 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 575/1144] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:14:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 14:14:48.991540 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7634c7d8-04af-413f-97e9-e75104b56e17 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:49.087675 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7634c7d8-04af-413f-97e9-e75104b56e17 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:49.119176 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57875905-7771-4792-8ce2-7d5828f74bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:49.119793 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57875905-7771-4792-8ce2-7d5828f74bbc None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:49.120069 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57875905-7771-4792-8ce2-7d5828f74bbc None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:49.120338 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57875905-7771-4792-8ce2-7d5828f74bbc None admin] PATH_INFO: `/v3/projects/55f8655b02134efcbd03a132ebd07e72` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:49.121433 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57875905-7771-4792-8ce2-7d5828f74bbc None admin] RBAC: Authorizing `identity:get_project(project_id=55f8655b02134efcbd03a132ebd07e72)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:49.122863 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57875905-7771-4792-8ce2-7d5828f74bbc None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:49.124588 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 570/1145] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:14:48 2022] GET /identity/v3/projects/55f8655b02134efcbd03a132ebd07e72 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:49.130333 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15995fd2-52d2-41cb-b827-50ba845b5b55 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:49.226050 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:49.232699 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15995fd2-52d2-41cb-b827-50ba845b5b55 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:49.265522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0798819b-5b1f-4a23-b75b-21dae570c1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:49.266117 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798819b-5b1f-4a23-b75b-21dae570c1c5 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:49.266398 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798819b-5b1f-4a23-b75b-21dae570c1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:49.266672 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798819b-5b1f-4a23-b75b-21dae570c1c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:49.267412 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0798819b-5b1f-4a23-b75b-21dae570c1c5 None admin] RBAC: Adding query filter params (role.id=d00b78c32dbf4cc7925a55bb621d8fcf, scope.project.id=55f8655b02134efcbd03a132ebd07e72, user.id=735ad65a54eb4670b9f0a6a011dc2e78) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:49.267719 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0798819b-5b1f-4a23-b75b-21dae570c1c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:49.269153 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0798819b-5b1f-4a23-b75b-21dae570c1c5 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:49.274874 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 576/1146] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:14:49 2022] GET /identity/v3/role_assignments?user.id=735ad65a54eb4670b9f0a6a011dc2e78&scope.project.id=55f8655b02134efcbd03a132ebd07e72&role.id=d00b78c32dbf4cc7925a55bb621d8fcf => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:51.350758 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd02df-df6a-4e0a-af9e-4f9e3ac0e840 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:51.351819 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd02df-df6a-4e0a-af9e-4f9e3ac0e840 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:51.351819 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd02df-df6a-4e0a-af9e-4f9e3ac0e840 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:51.353125 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 571/1147] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:14:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:14:51.358464 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a9e59-b132-4d52-b75b-26787d6b3d88 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:51.358858 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a9e59-b132-4d52-b75b-26787d6b3d88 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:51.359055 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a9e59-b132-4d52-b75b-26787d6b3d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:51.377507 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-a36a9e59-b132-4d52-b75b-26787d6b3d88 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:14:51.378850 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-a36a9e59-b132-4d52-b75b-26787d6b3d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:14:51.443354 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 577/1148] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:14:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:14:51.707688 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bd02df-df6a-4e0a-af9e-4f9e3ac0e840 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:51.737686 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:51.805816 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bd02df-df6a-4e0a-af9e-4f9e3ac0e840 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:51.838618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3466e4d-71b3-4f8e-9caa-b4c1f97a3e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:51.839273 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3466e4d-71b3-4f8e-9caa-b4c1f97a3e41 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:51.839546 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3466e4d-71b3-4f8e-9caa-b4c1f97a3e41 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:51.839810 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3466e4d-71b3-4f8e-9caa-b4c1f97a3e41 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:51.849395 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3466e4d-71b3-4f8e-9caa-b4c1f97a3e41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:14:51.854719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-a3466e4d-71b3-4f8e-9caa-b4c1f97a3e41 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:14:51.854719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:51.854719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:51.854719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:51.854719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:51.854719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:51.854719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:51.854719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:51.854719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:51.854719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:51.854719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:51.854719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:51.854719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:14:51.854719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:14:51.854719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:14:51.854719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:14:51.855816 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:14:51.855816 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:14:51.855816 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:14:51.855816 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:14:51.855816 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:51.855816 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:51.855816 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:14:51.855816 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:14:51.855816 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:14:51.855816 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:14:51.855816 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:14:51.855816 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:14:51.855816 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:14:51.855816 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:14:51.855816 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:14:51.855816 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:14:51.856857 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:14:51.856857 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:14:51.856857 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:14:51.856857 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:14:51.856857 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:14:51.856857 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:14:51.856857 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:14:51.856857 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:14:51.856857 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:14:51.856857 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:14:51.856857 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:14:51.856857 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:51.856857 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 572/1149] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 14:14:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:14:51.862191 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a9e59-b132-4d52-b75b-26787d6b3d88 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:51.947776 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:51.957813 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a9e59-b132-4d52-b75b-26787d6b3d88 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:51.990395 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e28e3-e13f-4179-bdbc-84e8acad50d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:51.991014 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e28e3-e13f-4179-bdbc-84e8acad50d0 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:51.991294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e28e3-e13f-4179-bdbc-84e8acad50d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:51.991593 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e28e3-e13f-4179-bdbc-84e8acad50d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:51.992248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4e28e3-e13f-4179-bdbc-84e8acad50d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:51.992550 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4e28e3-e13f-4179-bdbc-84e8acad50d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:51.993911 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4e28e3-e13f-4179-bdbc-84e8acad50d0 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:52.001580 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 578/1150] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:14:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:52.007795 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3466e4d-71b3-4f8e-9caa-b4c1f97a3e41 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:52.102237 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:52.112147 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3466e4d-71b3-4f8e-9caa-b4c1f97a3e41 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:52.144591 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66d592a-3838-4347-b94a-608d3f31091b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:52.145212 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d592a-3838-4347-b94a-608d3f31091b None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:52.145488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d592a-3838-4347-b94a-608d3f31091b None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:52.145757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d592a-3838-4347-b94a-608d3f31091b None admin] PATH_INFO: `/v3/users/735ad65a54eb4670b9f0a6a011dc2e78` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:52.146873 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66d592a-3838-4347-b94a-608d3f31091b None admin] RBAC: Authorizing `identity:get_user(user_id=735ad65a54eb4670b9f0a6a011dc2e78)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:52.148325 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66d592a-3838-4347-b94a-608d3f31091b None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:52.149881 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 573/1151] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:14:52 2022] GET /identity/v3/users/735ad65a54eb4670b9f0a6a011dc2e78 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:52.155677 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e28e3-e13f-4179-bdbc-84e8acad50d0 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:52.252057 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e28e3-e13f-4179-bdbc-84e8acad50d0 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:52.284605 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530f552-86c4-404d-85f9-de2e6722764d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:52.285218 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530f552-86c4-404d-85f9-de2e6722764d None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:52.285493 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530f552-86c4-404d-85f9-de2e6722764d None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:52.285761 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530f552-86c4-404d-85f9-de2e6722764d None admin] PATH_INFO: `/v3/projects/55f8655b02134efcbd03a132ebd07e72` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:52.286868 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4530f552-86c4-404d-85f9-de2e6722764d None admin] RBAC: Authorizing `identity:get_project(project_id=55f8655b02134efcbd03a132ebd07e72)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:52.288317 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4530f552-86c4-404d-85f9-de2e6722764d None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:52.290477 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 579/1152] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:14:52 2022] GET /identity/v3/projects/55f8655b02134efcbd03a132ebd07e72 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:52.296364 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66d592a-3838-4347-b94a-608d3f31091b None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:52.390528 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66d592a-3838-4347-b94a-608d3f31091b None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:52.423746 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4095ff-93a6-45cd-898d-c7ca9fb1dc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:52.424383 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4095ff-93a6-45cd-898d-c7ca9fb1dc61 None admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:52.424657 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4095ff-93a6-45cd-898d-c7ca9fb1dc61 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:52.424924 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4095ff-93a6-45cd-898d-c7ca9fb1dc61 None admin] PATH_INFO: `/v3/projects/55f8655b02134efcbd03a132ebd07e72/users/735ad65a54eb4670b9f0a6a011dc2e78/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:52.426756 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4095ff-93a6-45cd-898d-c7ca9fb1dc61 None admin] RBAC: Authorizing `identity:create_grant(project_id=55f8655b02134efcbd03a132ebd07e72, user_id=735ad65a54eb4670b9f0a6a011dc2e78, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:52.428181 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4095ff-93a6-45cd-898d-c7ca9fb1dc61 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:52.440688 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 574/1153] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 14:14:52 2022] PUT /identity/v3/projects/55f8655b02134efcbd03a132ebd07e72/users/735ad65a54eb4670b9f0a6a011dc2e78/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:14:54.556736 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53355695-a9bc-4d8c-9d1f-b0c106b958e5 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:54.557327 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53355695-a9bc-4d8c-9d1f-b0c106b958e5 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:54.557327 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53355695-a9bc-4d8c-9d1f-b0c106b958e5 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:54.558605 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 580/1154] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:14:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:14:54.563750 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a8b68-d1da-437a-afd1-7144b14506fd None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:54.564138 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a8b68-d1da-437a-afd1-7144b14506fd None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:54.564323 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a8b68-d1da-437a-afd1-7144b14506fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:54.582565 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-1b6a8b68-d1da-437a-afd1-7144b14506fd None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:14:54.583781 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-1b6a8b68-d1da-437a-afd1-7144b14506fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:14:54.596406 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:54.648642 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 575/1155] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:14:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:14:54.925217 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53355695-a9bc-4d8c-9d1f-b0c106b958e5 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:54.948738 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:55.027612 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53355695-a9bc-4d8c-9d1f-b0c106b958e5 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:55.059370 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d0ab5-4728-49e0-86a5-02b86022358a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:55.059987 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d0ab5-4728-49e0-86a5-02b86022358a None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:55.060262 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d0ab5-4728-49e0-86a5-02b86022358a None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:55.060531 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d0ab5-4728-49e0-86a5-02b86022358a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:55.071181 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d0ab5-4728-49e0-86a5-02b86022358a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:14:55.075901 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-070d0ab5-4728-49e0-86a5-02b86022358a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:14:55.075901 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:55.075901 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:55.075901 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:55.075901 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:55.075901 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:55.075901 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:55.075901 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:55.075901 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:55.075901 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:55.075901 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:55.075901 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:55.075901 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:14:55.075901 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:14:55.075901 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:14:55.075901 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:14:55.077065 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:14:55.077065 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:14:55.077065 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:14:55.077065 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:14:55.077065 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:55.077065 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:55.077065 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:14:55.077065 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:14:55.077065 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:14:55.077065 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:14:55.077065 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:14:55.077065 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:14:55.077065 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:14:55.077065 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:14:55.077065 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:14:55.077065 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:14:55.078316 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:14:55.078316 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:14:55.078316 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:14:55.078316 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:14:55.078316 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:14:55.078316 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:14:55.078316 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:14:55.078316 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:14:55.078316 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:14:55.078316 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:14:55.078316 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:14:55.078316 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:14:55.078316 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 581/1156] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 14:14:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:14:55.083058 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a8b68-d1da-437a-afd1-7144b14506fd None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:55.176744 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a8b68-d1da-437a-afd1-7144b14506fd None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:55.208839 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821aa9e4-c2ae-476a-8231-3b473bc32374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:55.209424 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821aa9e4-c2ae-476a-8231-3b473bc32374 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:55.209700 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821aa9e4-c2ae-476a-8231-3b473bc32374 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:55.209966 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821aa9e4-c2ae-476a-8231-3b473bc32374 None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:55.211207 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821aa9e4-c2ae-476a-8231-3b473bc32374 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:55.211502 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821aa9e4-c2ae-476a-8231-3b473bc32374 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:55.212848 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821aa9e4-c2ae-476a-8231-3b473bc32374 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:55.220453 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 576/1157] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:14:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:14:55.226118 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d0ab5-4728-49e0-86a5-02b86022358a None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:55.303162 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:55.322594 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d0ab5-4728-49e0-86a5-02b86022358a None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:55.353871 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa389087-c87a-46fa-913f-f0c81cf1d8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:55.354482 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa389087-c87a-46fa-913f-f0c81cf1d8e5 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:55.354756 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa389087-c87a-46fa-913f-f0c81cf1d8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:55.355050 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa389087-c87a-46fa-913f-f0c81cf1d8e5 None admin] PATH_INFO: `/v3/users/735ad65a54eb4670b9f0a6a011dc2e78` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:55.356130 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa389087-c87a-46fa-913f-f0c81cf1d8e5 None admin] RBAC: Authorizing `identity:get_user(user_id=735ad65a54eb4670b9f0a6a011dc2e78)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:55.357548 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa389087-c87a-46fa-913f-f0c81cf1d8e5 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:55.359127 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 582/1158] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:14:55 2022] GET /identity/v3/users/735ad65a54eb4670b9f0a6a011dc2e78 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:55.365041 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821aa9e4-c2ae-476a-8231-3b473bc32374 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:55.378004 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:55.470312 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821aa9e4-c2ae-476a-8231-3b473bc32374 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:55.502939 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4474a73-066b-4746-9603-d9265a34f7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:55.503534 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4474a73-066b-4746-9603-d9265a34f7e6 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:55.503811 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4474a73-066b-4746-9603-d9265a34f7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:55.504079 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4474a73-066b-4746-9603-d9265a34f7e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:55.567646 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4474a73-066b-4746-9603-d9265a34f7e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:55.569079 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4474a73-066b-4746-9603-d9265a34f7e6 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:55.579988 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:55.662866 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 577/1159] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:14:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 14:14:55.669242 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa389087-c87a-46fa-913f-f0c81cf1d8e5 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:55.765153 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa389087-c87a-46fa-913f-f0c81cf1d8e5 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:55.798376 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a564da-992d-4b1d-880d-1709601ff6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:55.799037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a564da-992d-4b1d-880d-1709601ff6ce None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:55.799319 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a564da-992d-4b1d-880d-1709601ff6ce None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:55.799587 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a564da-992d-4b1d-880d-1709601ff6ce None admin] PATH_INFO: `/v3/projects/55f8655b02134efcbd03a132ebd07e72` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:55.800672 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a564da-992d-4b1d-880d-1709601ff6ce None admin] RBAC: Authorizing `identity:get_project(project_id=55f8655b02134efcbd03a132ebd07e72)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:55.802094 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a564da-992d-4b1d-880d-1709601ff6ce None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:55.803725 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 583/1160] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:14:55 2022] GET /identity/v3/projects/55f8655b02134efcbd03a132ebd07e72 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:55.810247 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4474a73-066b-4746-9603-d9265a34f7e6 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:55.905577 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4474a73-066b-4746-9603-d9265a34f7e6 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:55.916639 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:55.939912 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf448c8e-af9c-467e-b3df-3d1246cadbf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:55.940504 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf448c8e-af9c-467e-b3df-3d1246cadbf4 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:55.940776 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf448c8e-af9c-467e-b3df-3d1246cadbf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:55.941044 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf448c8e-af9c-467e-b3df-3d1246cadbf4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:55.941755 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf448c8e-af9c-467e-b3df-3d1246cadbf4 None admin] RBAC: Adding query filter params (role.id=d00b78c32dbf4cc7925a55bb621d8fcf, scope.project.id=55f8655b02134efcbd03a132ebd07e72, user.id=735ad65a54eb4670b9f0a6a011dc2e78) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:55.942057 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf448c8e-af9c-467e-b3df-3d1246cadbf4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:55.943497 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf448c8e-af9c-467e-b3df-3d1246cadbf4 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:55.949057 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 578/1161] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:14:55 2022] GET /identity/v3/role_assignments?user.id=735ad65a54eb4670b9f0a6a011dc2e78&scope.project.id=55f8655b02134efcbd03a132ebd07e72&role.id=d00b78c32dbf4cc7925a55bb621d8fcf => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:14:58.089630 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba419e-4bde-4fa4-992c-1d0734699dac None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:58.090135 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba419e-4bde-4fa4-992c-1d0734699dac None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:58.090135 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba419e-4bde-4fa4-992c-1d0734699dac None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:58.091472 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 584/1162] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:14:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:14:58.096690 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889ce7c6-949c-4a43-8136-daf9a43d8254 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:58.097098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889ce7c6-949c-4a43-8136-daf9a43d8254 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:58.097228 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889ce7c6-949c-4a43-8136-daf9a43d8254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:58.114722 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-889ce7c6-949c-4a43-8136-daf9a43d8254 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:14:58.115909 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-889ce7c6-949c-4a43-8136-daf9a43d8254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:14:58.178420 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 579/1163] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:14:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:14:58.451172 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba419e-4bde-4fa4-992c-1d0734699dac None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:58.469847 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:58.552523 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba419e-4bde-4fa4-992c-1d0734699dac None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:58.585190 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e58fa-023a-499e-b5e2-a99a93ab2bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:58.585787 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e58fa-023a-499e-b5e2-a99a93ab2bd5 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:58.586064 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e58fa-023a-499e-b5e2-a99a93ab2bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:58.586334 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e58fa-023a-499e-b5e2-a99a93ab2bd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:58.650633 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66e58fa-023a-499e-b5e2-a99a93ab2bd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:58.652265 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66e58fa-023a-499e-b5e2-a99a93ab2bd5 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:58.668882 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:58.750267 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 585/1164] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:14:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 14:14:58.756111 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889ce7c6-949c-4a43-8136-daf9a43d8254 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:58.851201 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889ce7c6-949c-4a43-8136-daf9a43d8254 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:58.853461 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:58.884230 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad62054b-842e-4171-b79f-693ec0407c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:58.884846 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad62054b-842e-4171-b79f-693ec0407c09 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:58.885119 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad62054b-842e-4171-b79f-693ec0407c09 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:58.885385 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad62054b-842e-4171-b79f-693ec0407c09 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:58.891694 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad62054b-842e-4171-b79f-693ec0407c09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:58.893103 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad62054b-842e-4171-b79f-693ec0407c09 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:58.904602 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-ad62054b-842e-4171-b79f-693ec0407c09 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 24 14:14:58.904602 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:58.904602 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 24 14:14:58.904602 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 24 14:14:58.904602 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:14:58.904602 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:14:58.904602 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:14:58.904602 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:14:58.904602 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 24 14:14:58.904602 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:58.904602 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:14:58.904602 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:58.904602 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:14:58.904602 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:14:58.904602 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:14:58.904602 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:14:58.904602 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:14:58.905893 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:14:58.905893 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:14:58.905893 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:14:58.905893 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:14:58.905893 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:14:58.905893 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:14:58.905893 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:14:58.905893 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:14:58.905893 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:14:58.905893 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:14:58.905893 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:14:58.905893 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:14:58.905893 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:14:58.905893 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:14:58.905893 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:14:58.905893 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:14:58.907239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:14:58.907239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:14:58.907239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:14:58.907239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:14:58.907239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:14:58.907239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:14:58.907239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:14:58.907239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:14:58.907239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:14:58.907239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:14:58.907239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 24 14:14:58.907239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:14:58.907239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 24 14:14:58.907239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:14:58.907239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 580/1165] 10.222.0.52 () {56 vars in 1117 bytes} [Sat Sep 24 14:14:58 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:14:58.911869 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e58fa-023a-499e-b5e2-a99a93ab2bd5 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:14:59.008498 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e58fa-023a-499e-b5e2-a99a93ab2bd5 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:14:59.024675 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:14:59.042376 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca2ee58-9be8-4a0b-b296-bb97a9518267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:14:59.043009 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2ee58-9be8-4a0b-b296-bb97a9518267 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:14:59.043289 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2ee58-9be8-4a0b-b296-bb97a9518267 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:14:59.043560 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2ee58-9be8-4a0b-b296-bb97a9518267 None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:14:59.044193 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca2ee58-9be8-4a0b-b296-bb97a9518267 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:14:59.044493 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca2ee58-9be8-4a0b-b296-bb97a9518267 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:14:59.045860 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca2ee58-9be8-4a0b-b296-bb97a9518267 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:14:59.052822 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 586/1166] 10.222.0.52 () {56 vars in 1115 bytes} [Sat Sep 24 14:14:58 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:15:01.173530 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb3434f-90c8-43e6-9c80-b60a53a48059 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:15:01.174071 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb3434f-90c8-43e6-9c80-b60a53a48059 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:15:01.174071 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb3434f-90c8-43e6-9c80-b60a53a48059 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:15:01.175424 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 581/1167] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:15:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:15:01.180684 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888c9ed-54e1-4303-b6e0-3bd8e3f4d2f6 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:15:01.181183 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888c9ed-54e1-4303-b6e0-3bd8e3f4d2f6 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:15:01.181286 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888c9ed-54e1-4303-b6e0-3bd8e3f4d2f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:15:01.199389 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-f888c9ed-54e1-4303-b6e0-3bd8e3f4d2f6 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:15:01.200549 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-f888c9ed-54e1-4303-b6e0-3bd8e3f4d2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:15:01.264098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 587/1168] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:15:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:15:01.537949 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb3434f-90c8-43e6-9c80-b60a53a48059 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:15:01.636223 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb3434f-90c8-43e6-9c80-b60a53a48059 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:15:01.668286 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8624b4-9f83-480d-af20-f05a3f1d64dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:15:01.668878 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8624b4-9f83-480d-af20-f05a3f1d64dc None admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:15:01.669154 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8624b4-9f83-480d-af20-f05a3f1d64dc None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:15:01.669425 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8624b4-9f83-480d-af20-f05a3f1d64dc None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:15:01.670079 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8624b4-9f83-480d-af20-f05a3f1d64dc None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:15:01.671484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8624b4-9f83-480d-af20-f05a3f1d64dc None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:15:01.694277 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 582/1169] 10.222.0.52 () {60 vars in 1138 bytes} [Sat Sep 24 14:15:01 2022] POST /identity/v3/domains => generated 267 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:15:03.860685 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb7937-7882-4f32-b57e-d3395f0287c1 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:15:03.861137 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb7937-7882-4f32-b57e-d3395f0287c1 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:15:03.861237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb7937-7882-4f32-b57e-d3395f0287c1 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:15:03.862579 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 588/1170] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:15:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:15:03.867968 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e1df3-454f-4d74-a037-210a1dbd83c0 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:15:03.868351 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e1df3-454f-4d74-a037-210a1dbd83c0 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:15:03.868512 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e1df3-454f-4d74-a037-210a1dbd83c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:15:03.886197 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-c03e1df3-454f-4d74-a037-210a1dbd83c0 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:15:03.887398 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-c03e1df3-454f-4d74-a037-210a1dbd83c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:15:03.904686 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:15:03.953582 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 583/1171] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:15:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:15:04.226066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb7937-7882-4f32-b57e-d3395f0287c1 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:15:04.324158 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb7937-7882-4f32-b57e-d3395f0287c1 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:15:04.334207 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:15:04.357026 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039b1f81-ef67-466b-9c84-bd0f052e43b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:15:04.357639 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b1f81-ef67-466b-9c84-bd0f052e43b5 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:15:04.357912 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b1f81-ef67-466b-9c84-bd0f052e43b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:15:04.358177 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b1f81-ef67-466b-9c84-bd0f052e43b5 None admin] PATH_INFO: `/v3/domains/f4f489a79765447f95712bc61a6bc970` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:15:04.368584 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039b1f81-ef67-466b-9c84-bd0f052e43b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f4f489a79765447f95712bc61a6bc970)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:15:04.370009 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039b1f81-ef67-466b-9c84-bd0f052e43b5 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:15:04.371577 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 589/1172] 10.222.0.52 () {56 vars in 1183 bytes} [Sat Sep 24 14:15:04 2022] GET /identity/v3/domains/f4f489a79765447f95712bc61a6bc970 => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:15:04.377284 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e1df3-454f-4d74-a037-210a1dbd83c0 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:15:04.472201 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e1df3-454f-4d74-a037-210a1dbd83c0 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:15:04.504406 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add045e3-8c17-49e3-bc19-c1582b4a752a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:15:04.504939 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add045e3-8c17-49e3-bc19-c1582b4a752a None admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:15:04.505215 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add045e3-8c17-49e3-bc19-c1582b4a752a None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:15:04.505483 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add045e3-8c17-49e3-bc19-c1582b4a752a None admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:15:04.506131 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add045e3-8c17-49e3-bc19-c1582b4a752a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:15:04.507600 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add045e3-8c17-49e3-bc19-c1582b4a752a None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:15:04.532481 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 584/1173] 10.222.0.52 () {60 vars in 1141 bytes} [Sat Sep 24 14:15:04 2022] POST /identity/v3/projects => generated 361 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:15:06.687728 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8333336-4164-4997-8560-70bf60b22cc9 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:15:06.688253 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8333336-4164-4997-8560-70bf60b22cc9 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:15:06.688690 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8333336-4164-4997-8560-70bf60b22cc9 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:15:06.690312 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 590/1174] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:15:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:15:06.695758 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a062f-3e7e-417c-be52-d6c8bd26c526 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:15:06.696253 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a062f-3e7e-417c-be52-d6c8bd26c526 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:15:06.696724 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a062f-3e7e-417c-be52-d6c8bd26c526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:15:06.710477 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:15:06.716022 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-f62a062f-3e7e-417c-be52-d6c8bd26c526 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:15:06.717437 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-f62a062f-3e7e-417c-be52-d6c8bd26c526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:15:06.781340 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 585/1175] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:15:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:15:07.053311 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8333336-4164-4997-8560-70bf60b22cc9 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:15:07.150924 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8333336-4164-4997-8560-70bf60b22cc9 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:15:07.183945 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf5a55-0870-489f-a2db-578c9d9ee714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:15:07.184553 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf5a55-0870-489f-a2db-578c9d9ee714 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:15:07.184827 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf5a55-0870-489f-a2db-578c9d9ee714 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:15:07.185093 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf5a55-0870-489f-a2db-578c9d9ee714 None admin] PATH_INFO: `/v3/domains/f4f489a79765447f95712bc61a6bc970` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:15:07.186172 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf5a55-0870-489f-a2db-578c9d9ee714 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f4f489a79765447f95712bc61a6bc970)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:15:07.187619 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf5a55-0870-489f-a2db-578c9d9ee714 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:15:07.189169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 591/1176] 10.222.0.52 () {56 vars in 1183 bytes} [Sat Sep 24 14:15:07 2022] GET /identity/v3/domains/f4f489a79765447f95712bc61a6bc970 => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:15:07.195535 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a062f-3e7e-417c-be52-d6c8bd26c526 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:15:07.291242 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a062f-3e7e-417c-be52-d6c8bd26c526 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:15:07.324240 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8407491-2bf9-4604-824a-b75f7a2f7bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:15:07.324877 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8407491-2bf9-4604-824a-b75f7a2f7bdc None admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:15:07.325052 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8407491-2bf9-4604-824a-b75f7a2f7bdc None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:15:07.325324 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8407491-2bf9-4604-824a-b75f7a2f7bdc None admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:15:07.325972 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8407491-2bf9-4604-824a-b75f7a2f7bdc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:15:07.327431 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8407491-2bf9-4604-824a-b75f7a2f7bdc None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:15:07.336523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:15:07.349698 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 586/1177] 10.222.0.52 () {60 vars in 1132 bytes} [Sat Sep 24 14:15:07 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:15:09.525904 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bcbf3e-ea10-40e6-9b90-94259137bba6 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:15:09.526455 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bcbf3e-ea10-40e6-9b90-94259137bba6 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:15:09.526455 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bcbf3e-ea10-40e6-9b90-94259137bba6 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:15:09.527833 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 592/1178] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:15:09.532915 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df0372-590c-4284-a2d5-345869ce2ca3 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:15:09.533282 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df0372-590c-4284-a2d5-345869ce2ca3 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:15:09.533458 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df0372-590c-4284-a2d5-345869ce2ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:15:09.551300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-e9df0372-590c-4284-a2d5-345869ce2ca3 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:15:09.552464 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-e9df0372-590c-4284-a2d5-345869ce2ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:15:09.615280 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 587/1179] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:15:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:15:09.886943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bcbf3e-ea10-40e6-9b90-94259137bba6 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:15:09.927684 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:15:09.986627 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bcbf3e-ea10-40e6-9b90-94259137bba6 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:15:10.018556 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d53e3-9b7e-4ea5-8a4b-55420ed6182d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:15:10.019193 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d53e3-9b7e-4ea5-8a4b-55420ed6182d None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:15:10.019469 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d53e3-9b7e-4ea5-8a4b-55420ed6182d None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:15:10.019731 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d53e3-9b7e-4ea5-8a4b-55420ed6182d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:15:10.035615 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945d53e3-9b7e-4ea5-8a4b-55420ed6182d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:15:10.040396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-945d53e3-9b7e-4ea5-8a4b-55420ed6182d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:15:10.040396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:15:10.040396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:15:10.040396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:15:10.040396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:15:10.040396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:15:10.040396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:15:10.040396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:15:10.040396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:15:10.040396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:15:10.040396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:15:10.040396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:15:10.040396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:15:10.040396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:15:10.040396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:15:10.040396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:15:10.041526 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:15:10.041526 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:15:10.041526 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:15:10.041526 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:15:10.041526 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:15:10.041526 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:15:10.041526 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:15:10.041526 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:15:10.041526 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:15:10.041526 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:15:10.041526 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:15:10.041526 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:15:10.041526 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:15:10.041526 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:15:10.041526 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:15:10.041526 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:15:10.042527 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:15:10.042527 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:15:10.042527 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:15:10.042527 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:15:10.042527 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:15:10.042527 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:15:10.042527 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:15:10.042527 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:15:10.042527 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:15:10.042527 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:15:10.042527 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:15:10.042527 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:15:10.042527 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 593/1180] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 14:15:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:15:10.047604 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9df0372-590c-4284-a2d5-345869ce2ca3 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:15:10.151458 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9df0372-590c-4284-a2d5-345869ce2ca3 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:15:10.173459 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:15:10.185307 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4f7c9-5848-42f6-bb80-bd08fc0ebeee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:15:10.185899 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4f7c9-5848-42f6-bb80-bd08fc0ebeee None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:15:10.186172 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4f7c9-5848-42f6-bb80-bd08fc0ebeee None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:15:10.186438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4f7c9-5848-42f6-bb80-bd08fc0ebeee None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:15:10.187113 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d4f7c9-5848-42f6-bb80-bd08fc0ebeee None admin] RBAC: Adding query filter params (name=admin) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:15:10.187407 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d4f7c9-5848-42f6-bb80-bd08fc0ebeee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:15:10.188756 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d4f7c9-5848-42f6-bb80-bd08fc0ebeee None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:15:10.196878 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 588/1181] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:15:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:15:10.202767 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d53e3-9b7e-4ea5-8a4b-55420ed6182d None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:15:10.296442 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:15:10.300526 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d53e3-9b7e-4ea5-8a4b-55420ed6182d None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:15:10.333261 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d0597e-2e0b-4c74-afba-b6c6012e6aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:15:10.333873 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0597e-2e0b-4c74-afba-b6c6012e6aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:15:10.334151 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0597e-2e0b-4c74-afba-b6c6012e6aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:15:10.334421 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0597e-2e0b-4c74-afba-b6c6012e6aa8 None admin] PATH_INFO: `/v3/users/759e291f190d4424a2c842eaa721bd39` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:15:10.347407 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d0597e-2e0b-4c74-afba-b6c6012e6aa8 None admin] RBAC: Authorizing `identity:get_user(user_id=759e291f190d4424a2c842eaa721bd39)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:15:10.348875 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d0597e-2e0b-4c74-afba-b6c6012e6aa8 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:15:10.350553 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 594/1182] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:15:10 2022] GET /identity/v3/users/759e291f190d4424a2c842eaa721bd39 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:15:10.356438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4f7c9-5848-42f6-bb80-bd08fc0ebeee None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:15:10.451985 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4f7c9-5848-42f6-bb80-bd08fc0ebeee None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:15:10.482869 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e0b105-1802-4272-9bc4-b5b8b1067542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:15:10.483451 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0b105-1802-4272-9bc4-b5b8b1067542 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:15:10.483726 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0b105-1802-4272-9bc4-b5b8b1067542 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:15:10.483988 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0b105-1802-4272-9bc4-b5b8b1067542 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:15:10.533129 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:15:10.548545 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e0b105-1802-4272-9bc4-b5b8b1067542 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:15:10.549975 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e0b105-1802-4272-9bc4-b5b8b1067542 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:15:10.642773 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 589/1183] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:15:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 14:15:10.648750 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d0597e-2e0b-4c74-afba-b6c6012e6aa8 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:15:10.744371 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d0597e-2e0b-4c74-afba-b6c6012e6aa8 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:15:10.776561 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df947a-dc97-42db-910e-fc62e417cfa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:15:10.777173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df947a-dc97-42db-910e-fc62e417cfa0 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:15:10.777446 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df947a-dc97-42db-910e-fc62e417cfa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:15:10.777713 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df947a-dc97-42db-910e-fc62e417cfa0 None admin] PATH_INFO: `/v3/projects/fe99d918fc894a8dbed251dbf7ec2cd4` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:15:10.778795 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90df947a-dc97-42db-910e-fc62e417cfa0 None admin] RBAC: Authorizing `identity:get_project(project_id=fe99d918fc894a8dbed251dbf7ec2cd4)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:15:10.780235 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90df947a-dc97-42db-910e-fc62e417cfa0 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:15:10.781811 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 595/1184] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:15:10 2022] GET /identity/v3/projects/fe99d918fc894a8dbed251dbf7ec2cd4 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:15:10.788617 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e0b105-1802-4272-9bc4-b5b8b1067542 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:15:10.869361 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:15:10.885092 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e0b105-1802-4272-9bc4-b5b8b1067542 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:15:10.917913 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c9b98-1167-4818-be12-47f9e80e202f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:15:10.918510 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c9b98-1167-4818-be12-47f9e80e202f None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:15:10.918788 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c9b98-1167-4818-be12-47f9e80e202f None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:15:10.919093 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c9b98-1167-4818-be12-47f9e80e202f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:15:10.919808 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7c9b98-1167-4818-be12-47f9e80e202f None admin] RBAC: Adding query filter params (role.id=d00b78c32dbf4cc7925a55bb621d8fcf, scope.project.id=fe99d918fc894a8dbed251dbf7ec2cd4, user.id=759e291f190d4424a2c842eaa721bd39) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:15:10.920114 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7c9b98-1167-4818-be12-47f9e80e202f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:15:10.921534 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7c9b98-1167-4818-be12-47f9e80e202f None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:15:10.926688 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 590/1185] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:15:10 2022] GET /identity/v3/role_assignments?user.id=759e291f190d4424a2c842eaa721bd39&scope.project.id=fe99d918fc894a8dbed251dbf7ec2cd4&role.id=d00b78c32dbf4cc7925a55bb621d8fcf => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:15:13.063031 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb77dc-ec53-4096-8291-1438ab661a02 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:15:13.063568 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb77dc-ec53-4096-8291-1438ab661a02 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:15:13.063568 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb77dc-ec53-4096-8291-1438ab661a02 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:15:13.064909 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 596/1186] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:15:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:15:13.070608 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b287f-b00a-4bf8-93f0-8eedbe32e060 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:15:13.071048 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b287f-b00a-4bf8-93f0-8eedbe32e060 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:15:13.071187 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b287f-b00a-4bf8-93f0-8eedbe32e060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:15:13.089650 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-4f5b287f-b00a-4bf8-93f0-8eedbe32e060 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:15:13.090754 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-4f5b287f-b00a-4bf8-93f0-8eedbe32e060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:15:13.154615 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 591/1187] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:15:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:15:13.420761 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb77dc-ec53-4096-8291-1438ab661a02 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:15:13.446176 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:15:13.520732 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb77dc-ec53-4096-8291-1438ab661a02 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:15:13.553907 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f5190-69ac-4a89-9f83-03146fe9f381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:15:13.554522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f5190-69ac-4a89-9f83-03146fe9f381 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:15:13.554794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f5190-69ac-4a89-9f83-03146fe9f381 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:15:13.555087 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f5190-69ac-4a89-9f83-03146fe9f381 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:15:13.564843 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771f5190-69ac-4a89-9f83-03146fe9f381 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:15:13.570149 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-771f5190-69ac-4a89-9f83-03146fe9f381 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:15:13.570149 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:15:13.570149 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:15:13.570149 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:15:13.570149 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:15:13.570149 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:15:13.570149 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:15:13.570149 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:15:13.570149 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:15:13.570149 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:15:13.570149 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:15:13.570149 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:15:13.570149 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:15:13.570149 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:15:13.570149 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:15:13.570149 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:15:13.571260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:15:13.571260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:15:13.571260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:15:13.571260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:15:13.571260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:15:13.571260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:15:13.571260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:15:13.571260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:15:13.571260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:15:13.571260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:15:13.571260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:15:13.571260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:15:13.571260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:15:13.571260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:15:13.571260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:15:13.571260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:15:13.572294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:15:13.572294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:15:13.572294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:15:13.572294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:15:13.572294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:15:13.572294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:15:13.572294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:15:13.572294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:15:13.572294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:15:13.572294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:15:13.572294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:15:13.572294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:15:13.572294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 597/1188] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 14:15:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:15:13.577300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b287f-b00a-4bf8-93f0-8eedbe32e060 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:15:13.650981 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:15:13.672268 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b287f-b00a-4bf8-93f0-8eedbe32e060 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:15:13.704310 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd230a78-9529-45f2-a0fd-fe960d2a9d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:15:13.704902 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd230a78-9529-45f2-a0fd-fe960d2a9d66 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:15:13.705175 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd230a78-9529-45f2-a0fd-fe960d2a9d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:15:13.705445 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd230a78-9529-45f2-a0fd-fe960d2a9d66 None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:15:13.706098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd230a78-9529-45f2-a0fd-fe960d2a9d66 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:15:13.706399 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd230a78-9529-45f2-a0fd-fe960d2a9d66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:15:13.707781 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd230a78-9529-45f2-a0fd-fe960d2a9d66 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:15:13.715432 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 592/1189] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:15:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:15:13.721105 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f5190-69ac-4a89-9f83-03146fe9f381 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:15:13.803604 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:15:13.824066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f5190-69ac-4a89-9f83-03146fe9f381 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:15:13.857950 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e47097-26d5-465f-88b6-e5e9a82907da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:15:13.858579 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e47097-26d5-465f-88b6-e5e9a82907da None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:15:13.858882 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e47097-26d5-465f-88b6-e5e9a82907da None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:15:13.859156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e47097-26d5-465f-88b6-e5e9a82907da None admin] PATH_INFO: `/v3/users/759e291f190d4424a2c842eaa721bd39` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:15:13.860260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e47097-26d5-465f-88b6-e5e9a82907da None admin] RBAC: Authorizing `identity:get_user(user_id=759e291f190d4424a2c842eaa721bd39)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:15:13.861697 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e47097-26d5-465f-88b6-e5e9a82907da None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:15:13.863301 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 598/1190] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:15:13 2022] GET /identity/v3/users/759e291f190d4424a2c842eaa721bd39 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:15:13.869116 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd230a78-9529-45f2-a0fd-fe960d2a9d66 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:15:13.965047 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd230a78-9529-45f2-a0fd-fe960d2a9d66 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:15:13.996194 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa775af7-1cec-4b11-85f0-c4a2498ae4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:15:13.996802 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa775af7-1cec-4b11-85f0-c4a2498ae4bc None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:15:13.997075 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa775af7-1cec-4b11-85f0-c4a2498ae4bc None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:15:13.997337 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa775af7-1cec-4b11-85f0-c4a2498ae4bc None admin] PATH_INFO: `/v3/projects/fe99d918fc894a8dbed251dbf7ec2cd4` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:15:13.998416 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa775af7-1cec-4b11-85f0-c4a2498ae4bc None admin] RBAC: Authorizing `identity:get_project(project_id=fe99d918fc894a8dbed251dbf7ec2cd4)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:15:13.999847 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa775af7-1cec-4b11-85f0-c4a2498ae4bc None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:15:14.001423 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 593/1191] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:15:13 2022] GET /identity/v3/projects/fe99d918fc894a8dbed251dbf7ec2cd4 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:15:14.007810 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e47097-26d5-465f-88b6-e5e9a82907da None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:15:14.106506 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e47097-26d5-465f-88b6-e5e9a82907da None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:15:14.139158 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17adf8c3-131e-45d8-941e-713f99eaa3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:15:14.139805 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17adf8c3-131e-45d8-941e-713f99eaa3d3 None admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:15:14.140086 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17adf8c3-131e-45d8-941e-713f99eaa3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:15:14.140359 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17adf8c3-131e-45d8-941e-713f99eaa3d3 None admin] PATH_INFO: `/v3/projects/fe99d918fc894a8dbed251dbf7ec2cd4/users/759e291f190d4424a2c842eaa721bd39/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:15:14.142232 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17adf8c3-131e-45d8-941e-713f99eaa3d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=fe99d918fc894a8dbed251dbf7ec2cd4, user_id=759e291f190d4424a2c842eaa721bd39, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:15:14.143702 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17adf8c3-131e-45d8-941e-713f99eaa3d3 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:15:14.156905 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 599/1192] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 14:15:14 2022] PUT /identity/v3/projects/fe99d918fc894a8dbed251dbf7ec2cd4/users/759e291f190d4424a2c842eaa721bd39/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:15:16.263320 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d6813-4f4b-4204-8741-7a33161487ad None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:15:16.263850 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d6813-4f4b-4204-8741-7a33161487ad None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:15:16.263850 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d6813-4f4b-4204-8741-7a33161487ad None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:15:16.265181 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 594/1193] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:15:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:15:16.270237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7bf255-8dda-4a5c-8d2d-0dd0806914c7 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:15:16.270632 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7bf255-8dda-4a5c-8d2d-0dd0806914c7 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:15:16.270777 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7bf255-8dda-4a5c-8d2d-0dd0806914c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:15:16.289632 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-fe7bf255-8dda-4a5c-8d2d-0dd0806914c7 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:15:16.290835 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-fe7bf255-8dda-4a5c-8d2d-0dd0806914c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:15:16.297907 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:15:16.356657 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 600/1194] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:15:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:15:16.633645 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d6813-4f4b-4204-8741-7a33161487ad None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:15:16.644899 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:15:16.731629 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d6813-4f4b-4204-8741-7a33161487ad None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:15:16.771027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703dcf49-6c80-41a6-8bea-d016b510b3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:15:16.772168 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703dcf49-6c80-41a6-8bea-d016b510b3ca None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:15:16.772664 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703dcf49-6c80-41a6-8bea-d016b510b3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:15:16.773077 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703dcf49-6c80-41a6-8bea-d016b510b3ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:15:16.788310 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703dcf49-6c80-41a6-8bea-d016b510b3ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:15:16.793027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-703dcf49-6c80-41a6-8bea-d016b510b3ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:15:16.793027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:15:16.793027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:15:16.793027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:15:16.793027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:15:16.793027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:15:16.793027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:15:16.793027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:15:16.793027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:15:16.793027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:15:16.793027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:15:16.793027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:15:16.793027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:15:16.793027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:15:16.793027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:15:16.793027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:15:16.794143 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:15:16.794143 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:15:16.794143 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:15:16.794143 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:15:16.794143 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:15:16.794143 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:15:16.794143 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:15:16.794143 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:15:16.794143 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:15:16.794143 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:15:16.794143 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:15:16.794143 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:15:16.794143 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:15:16.794143 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:15:16.794143 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:15:16.794143 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:15:16.795189 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:15:16.795189 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:15:16.795189 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:15:16.795189 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:15:16.795189 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:15:16.795189 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:15:16.795189 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:15:16.795189 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:15:16.795189 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:15:16.795189 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:15:16.795189 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:15:16.795189 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:15:16.795189 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 595/1195] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 14:15:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:15:16.800248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7bf255-8dda-4a5c-8d2d-0dd0806914c7 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:15:16.898693 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7bf255-8dda-4a5c-8d2d-0dd0806914c7 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:15:16.932289 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1f1ef-fa4d-4730-99c8-2287fc9830a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:15:16.932881 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1f1ef-fa4d-4730-99c8-2287fc9830a8 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:15:16.933160 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1f1ef-fa4d-4730-99c8-2287fc9830a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:15:16.933433 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1f1ef-fa4d-4730-99c8-2287fc9830a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:15:16.934094 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d1f1ef-fa4d-4730-99c8-2287fc9830a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:15:16.934398 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d1f1ef-fa4d-4730-99c8-2287fc9830a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:15:16.935795 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d1f1ef-fa4d-4730-99c8-2287fc9830a8 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:15:16.943662 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 601/1196] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:15:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:15:16.949527 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703dcf49-6c80-41a6-8bea-d016b510b3ca None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:15:17.016434 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:15:17.049264 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703dcf49-6c80-41a6-8bea-d016b510b3ca None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:15:17.082533 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7f98c6-27ca-4725-a420-740496293a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:15:17.083196 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7f98c6-27ca-4725-a420-740496293a2d None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:15:17.083481 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7f98c6-27ca-4725-a420-740496293a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:15:17.083753 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7f98c6-27ca-4725-a420-740496293a2d None admin] PATH_INFO: `/v3/users/759e291f190d4424a2c842eaa721bd39` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:15:17.085432 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7f98c6-27ca-4725-a420-740496293a2d None admin] RBAC: Authorizing `identity:get_user(user_id=759e291f190d4424a2c842eaa721bd39)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:15:17.086886 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7f98c6-27ca-4725-a420-740496293a2d None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:15:17.088508 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 596/1197] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 14:15:16 2022] GET /identity/v3/users/759e291f190d4424a2c842eaa721bd39 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:15:17.094993 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1f1ef-fa4d-4730-99c8-2287fc9830a8 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:15:17.099560 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:15:17.192389 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1f1ef-fa4d-4730-99c8-2287fc9830a8 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:15:17.224077 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed63077-8af1-4bc3-98db-eed2229c6542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:15:17.224672 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed63077-8af1-4bc3-98db-eed2229c6542 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:15:17.225485 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed63077-8af1-4bc3-98db-eed2229c6542 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:15:17.225754 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed63077-8af1-4bc3-98db-eed2229c6542 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:15:17.289580 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed63077-8af1-4bc3-98db-eed2229c6542 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:15:17.291035 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed63077-8af1-4bc3-98db-eed2229c6542 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:15:17.293499 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:15:17.385380 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 602/1198] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:15:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 14:15:17.391317 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7f98c6-27ca-4725-a420-740496293a2d None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:15:17.485921 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7f98c6-27ca-4725-a420-740496293a2d None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:15:17.519039 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f80f32-eb4b-46ee-8268-8d043ba07bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:15:17.519679 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f80f32-eb4b-46ee-8268-8d043ba07bc6 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:15:17.519953 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f80f32-eb4b-46ee-8268-8d043ba07bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:15:17.520218 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f80f32-eb4b-46ee-8268-8d043ba07bc6 None admin] PATH_INFO: `/v3/projects/fe99d918fc894a8dbed251dbf7ec2cd4` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:15:17.521304 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f80f32-eb4b-46ee-8268-8d043ba07bc6 None admin] RBAC: Authorizing `identity:get_project(project_id=fe99d918fc894a8dbed251dbf7ec2cd4)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:15:17.523337 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f80f32-eb4b-46ee-8268-8d043ba07bc6 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:15:17.524918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 597/1199] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 14:15:17 2022] GET /identity/v3/projects/fe99d918fc894a8dbed251dbf7ec2cd4 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:15:17.530886 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed63077-8af1-4bc3-98db-eed2229c6542 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:15:17.625702 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed63077-8af1-4bc3-98db-eed2229c6542 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:15:17.627985 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:15:17.659680 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d490b3-cad1-4263-95d0-ee37949137be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:15:17.660274 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d490b3-cad1-4263-95d0-ee37949137be None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:15:17.660551 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d490b3-cad1-4263-95d0-ee37949137be None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:15:17.660817 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d490b3-cad1-4263-95d0-ee37949137be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:15:17.661529 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d490b3-cad1-4263-95d0-ee37949137be None admin] RBAC: Adding query filter params (role.id=d00b78c32dbf4cc7925a55bb621d8fcf, scope.project.id=fe99d918fc894a8dbed251dbf7ec2cd4, user.id=759e291f190d4424a2c842eaa721bd39) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:15:17.661834 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d490b3-cad1-4263-95d0-ee37949137be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:15:17.663349 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d490b3-cad1-4263-95d0-ee37949137be None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:15:17.668969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 603/1200] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:15:17 2022] GET /identity/v3/role_assignments?user.id=759e291f190d4424a2c842eaa721bd39&scope.project.id=fe99d918fc894a8dbed251dbf7ec2cd4&role.id=d00b78c32dbf4cc7925a55bb621d8fcf => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:15:27.515027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=98 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:15:27.666407 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=16 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:13.682708 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6face47a-d7f9-4899-9434-4211384dd6f8 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:13.682708 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6face47a-d7f9-4899-9434-4211384dd6f8 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:13.682708 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6face47a-d7f9-4899-9434-4211384dd6f8 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:13.682708 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 598/1201] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:16:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:16:13.689410 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3c246-49b6-4db4-9e03-5f20a528a696 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:13.689936 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3c246-49b6-4db4-9e03-5f20a528a696 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:13.689936 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3c246-49b6-4db4-9e03-5f20a528a696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:13.712196 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-90f3c246-49b6-4db4-9e03-5f20a528a696 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:16:13.713498 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-90f3c246-49b6-4db4-9e03-5f20a528a696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:16:13.812869 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 604/1202] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:16:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:16:14.128427 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6face47a-d7f9-4899-9434-4211384dd6f8 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:14.267047 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6face47a-d7f9-4899-9434-4211384dd6f8 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:14.313306 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b3a0a-d9fd-4b90-bd54-d69a8894e884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:14.314089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b3a0a-d9fd-4b90-bd54-d69a8894e884 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:14.314633 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b3a0a-d9fd-4b90-bd54-d69a8894e884 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:14.314756 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b3a0a-d9fd-4b90-bd54-d69a8894e884 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:14.328669 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50b3a0a-d9fd-4b90-bd54-d69a8894e884 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:14.330255 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50b3a0a-d9fd-4b90-bd54-d69a8894e884 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:14.348634 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-d50b3a0a-d9fd-4b90-bd54-d69a8894e884 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:16:14.348634 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:16:14.348634 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:16:14.348634 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:16:14.348634 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:16:14.348634 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:16:14.348634 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:16:14.348634 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:16:14.348634 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:16:14.348634 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:16:14.348634 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:16:14.348634 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:16:14.348634 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:16:14.348634 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:16:14.348634 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:16:14.348634 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:16:14.349765 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:16:14.349765 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:16:14.349765 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:16:14.349765 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:16:14.349765 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:16:14.349765 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:16:14.349765 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:16:14.349765 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:16:14.349765 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:16:14.349765 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:16:14.349765 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:16:14.349765 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:16:14.349765 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:16:14.349765 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:16:14.349765 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:16:14.349765 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:16:14.349765 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:16:14.350760 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:16:14.350760 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:16:14.350760 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:16:14.358059 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3c246-49b6-4db4-9e03-5f20a528a696 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:14.358216 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 599/1203] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:16:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:16:14.487128 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3c246-49b6-4db4-9e03-5f20a528a696 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:14.525052 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:14.532846 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ce418-a6dc-4ff1-9ce5-20d9ff6e5371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:14.534342 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ce418-a6dc-4ff1-9ce5-20d9ff6e5371 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:14.534342 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ce418-a6dc-4ff1-9ce5-20d9ff6e5371 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:14.534625 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ce418-a6dc-4ff1-9ce5-20d9ff6e5371 None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:14.535843 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5ce418-a6dc-4ff1-9ce5-20d9ff6e5371 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:16:14.536169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5ce418-a6dc-4ff1-9ce5-20d9ff6e5371 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:14.537563 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5ce418-a6dc-4ff1-9ce5-20d9ff6e5371 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:14.556813 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 605/1204] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:16:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:16:14.557545 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b3a0a-d9fd-4b90-bd54-d69a8894e884 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:14.628769 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:14.685473 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b3a0a-d9fd-4b90-bd54-d69a8894e884 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:14.721727 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2e5cc8-f6bf-4c35-94f0-79ed8146ad81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:14.722338 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e5cc8-f6bf-4c35-94f0-79ed8146ad81 None admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:14.723661 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e5cc8-f6bf-4c35-94f0-79ed8146ad81 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:14.723921 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e5cc8-f6bf-4c35-94f0-79ed8146ad81 None admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:14.724578 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2e5cc8-f6bf-4c35-94f0-79ed8146ad81 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:14.725999 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2e5cc8-f6bf-4c35-94f0-79ed8146ad81 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:14.753343 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 600/1205] 10.222.0.52 () {60 vars in 1132 bytes} [Sat Sep 24 14:16:14 2022] POST /identity/v3/users => generated 251 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:16:17.158142 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80d1bf-4df4-49c4-934d-54c777750453 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:17.159012 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80d1bf-4df4-49c4-934d-54c777750453 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:17.159012 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80d1bf-4df4-49c4-934d-54c777750453 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:17.160346 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 606/1206] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:16:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:16:17.166015 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4159d-435b-402c-bcfd-86c5d041f3e6 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:17.166483 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4159d-435b-402c-bcfd-86c5d041f3e6 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:17.166679 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4159d-435b-402c-bcfd-86c5d041f3e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:17.187925 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-60c4159d-435b-402c-bcfd-86c5d041f3e6 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:16:17.189142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-60c4159d-435b-402c-bcfd-86c5d041f3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:16:17.269693 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 601/1207] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:16:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:16:17.562587 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d80d1bf-4df4-49c4-934d-54c777750453 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:17.686124 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d80d1bf-4df4-49c4-934d-54c777750453 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:17.724510 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea7eaf-756e-44a0-b4dd-a43b32de3ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:17.725462 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea7eaf-756e-44a0-b4dd-a43b32de3ef5 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:17.725761 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea7eaf-756e-44a0-b4dd-a43b32de3ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:17.726008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea7eaf-756e-44a0-b4dd-a43b32de3ef5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:17.740496 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ea7eaf-756e-44a0-b4dd-a43b32de3ef5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:16:17.745913 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-82ea7eaf-756e-44a0-b4dd-a43b32de3ef5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:16:17.745913 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:16:17.745913 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:16:17.745913 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:16:17.745913 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:16:17.745913 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:16:17.745913 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:16:17.745913 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:16:17.745913 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:16:17.745913 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:16:17.745913 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:16:17.745913 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:16:17.745913 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:16:17.745913 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:16:17.745913 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:16:17.745913 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:16:17.751983 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:16:17.751983 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:16:17.751983 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:16:17.751983 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:16:17.751983 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:16:17.751983 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:16:17.751983 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:16:17.751983 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:16:17.751983 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:16:17.751983 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:16:17.751983 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:16:17.751983 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:16:17.751983 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:16:17.751983 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:16:17.751983 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:16:17.751983 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:16:17.753131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:16:17.753131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:16:17.753131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:16:17.753131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:16:17.753131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:16:17.753131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:16:17.753131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:16:17.753131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:16:17.753131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:16:17.753131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:16:17.753131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:16:17.753131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:16:17.753131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 607/1208] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 14:16:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:16:17.754843 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c4159d-435b-402c-bcfd-86c5d041f3e6 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:17.798383 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:17.864731 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c4159d-435b-402c-bcfd-86c5d041f3e6 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:17.903911 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1867c8c-870e-40e4-bf34-cb9c4d04fde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:17.904523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1867c8c-870e-40e4-bf34-cb9c4d04fde7 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:17.904808 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1867c8c-870e-40e4-bf34-cb9c4d04fde7 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:17.905060 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1867c8c-870e-40e4-bf34-cb9c4d04fde7 None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:17.905730 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1867c8c-870e-40e4-bf34-cb9c4d04fde7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:16:17.906349 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1867c8c-870e-40e4-bf34-cb9c4d04fde7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:17.907403 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1867c8c-870e-40e4-bf34-cb9c4d04fde7 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:17.916275 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 602/1209] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 14:16:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:16:17.923162 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea7eaf-756e-44a0-b4dd-a43b32de3ef5 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:17.988713 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:18.050210 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea7eaf-756e-44a0-b4dd-a43b32de3ef5 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:18.091546 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61424fe7-5a09-47a3-8ef1-266e51a21ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:18.091546 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61424fe7-5a09-47a3-8ef1-266e51a21ddc None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:18.091546 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61424fe7-5a09-47a3-8ef1-266e51a21ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:18.091906 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61424fe7-5a09-47a3-8ef1-266e51a21ddc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:18.104068 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61424fe7-5a09-47a3-8ef1-266e51a21ddc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:18.105684 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61424fe7-5a09-47a3-8ef1-266e51a21ddc None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:18.121271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-61424fe7-5a09-47a3-8ef1-266e51a21ddc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:16:18.121271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:16:18.121271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:16:18.121271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:16:18.121271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:16:18.121271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:16:18.121271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:16:18.121271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:16:18.121271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:16:18.121271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:16:18.121271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:16:18.121271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:16:18.121271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:16:18.121271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:16:18.121271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:16:18.121271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:16:18.122541 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:16:18.122541 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:16:18.122541 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:16:18.122541 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:16:18.122541 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:16:18.122541 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:16:18.122541 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:16:18.122541 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:16:18.122541 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:16:18.122541 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:16:18.122541 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:16:18.122541 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:16:18.122541 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:16:18.122541 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:16:18.122541 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:16:18.122541 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:16:18.122541 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:16:18.123698 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:16:18.123698 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:16:18.123698 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:16:18.125159 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 608/1210] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:16:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:16:18.131974 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1867c8c-870e-40e4-bf34-cb9c4d04fde7 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:18.253001 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:18.256629 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1867c8c-870e-40e4-bf34-cb9c4d04fde7 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:18.305868 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0324833-1765-4e83-a4ac-e533eb1a3b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:18.306775 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0324833-1765-4e83-a4ac-e533eb1a3b0c None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:18.307088 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0324833-1765-4e83-a4ac-e533eb1a3b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:18.307389 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0324833-1765-4e83-a4ac-e533eb1a3b0c None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:18.308129 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0324833-1765-4e83-a4ac-e533eb1a3b0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:16:18.308464 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0324833-1765-4e83-a4ac-e533eb1a3b0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:18.310077 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0324833-1765-4e83-a4ac-e533eb1a3b0c None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:18.324714 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 603/1211] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:16:18.332047 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61424fe7-5a09-47a3-8ef1-266e51a21ddc None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:18.465645 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61424fe7-5a09-47a3-8ef1-266e51a21ddc None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:18.467505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:18.509334 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b17002-afd3-450d-833b-d76b9dc027d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:18.510038 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b17002-afd3-450d-833b-d76b9dc027d8 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:18.510326 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b17002-afd3-450d-833b-d76b9dc027d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:18.510643 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b17002-afd3-450d-833b-d76b9dc027d8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:18.520532 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b17002-afd3-450d-833b-d76b9dc027d8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:18.522031 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b17002-afd3-450d-833b-d76b9dc027d8 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:18.537184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-95b17002-afd3-450d-833b-d76b9dc027d8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 24 14:16:18.537184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:16:18.537184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:16:18.537184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:16:18.537184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:16:18.537184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:16:18.537184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:16:18.537184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:16:18.537184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:16:18.537184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:16:18.537184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:16:18.537184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:16:18.537184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:16:18.537184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:16:18.537184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:16:18.537184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:16:18.538845 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:16:18.538845 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:16:18.538845 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:16:18.538845 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:16:18.538845 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:16:18.538845 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:16:18.538845 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:16:18.538845 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:16:18.538845 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:16:18.538845 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:16:18.538845 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:16:18.538845 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:16:18.538845 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:16:18.538845 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:16:18.538845 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:16:18.538845 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:16:18.538845 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:16:18.538845 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:16:18.539949 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:16:18.539949 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:16:18.539949 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:16:18.539949 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:16:18.539949 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:16:18.539949 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:16:18.539949 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:16:18.539949 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:16:18.539949 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 24 14:16:18.539949 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:16:18.539949 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 609/1212] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:16:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:16:18.545296 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0324833-1765-4e83-a4ac-e533eb1a3b0c None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:18.653432 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0324833-1765-4e83-a4ac-e533eb1a3b0c None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:18.701705 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4dcc6e-6e59-498c-a95c-517818ffef63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:18.703751 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4dcc6e-6e59-498c-a95c-517818ffef63 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:18.703751 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4dcc6e-6e59-498c-a95c-517818ffef63 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:18.703751 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4dcc6e-6e59-498c-a95c-517818ffef63 None admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:18.703751 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4dcc6e-6e59-498c-a95c-517818ffef63 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:16:18.704063 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4dcc6e-6e59-498c-a95c-517818ffef63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:18.705561 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4dcc6e-6e59-498c-a95c-517818ffef63 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:18.725469 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 604/1213] 10.222.0.52 () {56 vars in 1143 bytes} [Sat Sep 24 14:16:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:16:18.730923 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b17002-afd3-450d-833b-d76b9dc027d8 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:18.857365 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b17002-afd3-450d-833b-d76b9dc027d8 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:18.900870 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5696dd7-6bb8-4026-aa0e-4a74f9e4bbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:18.900870 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5696dd7-6bb8-4026-aa0e-4a74f9e4bbac None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:18.900870 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5696dd7-6bb8-4026-aa0e-4a74f9e4bbac None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:18.901283 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5696dd7-6bb8-4026-aa0e-4a74f9e4bbac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:18.934756 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:18.985304 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5696dd7-6bb8-4026-aa0e-4a74f9e4bbac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:18.987207 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5696dd7-6bb8-4026-aa0e-4a74f9e4bbac None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:19.115646 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 610/1214] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:16:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 14:16:19.116815 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4dcc6e-6e59-498c-a95c-517818ffef63 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:19.149731 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:19.257583 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4dcc6e-6e59-498c-a95c-517818ffef63 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:19.299020 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a814008-c6f4-44d0-b9f4-137db4741fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:19.300042 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a814008-c6f4-44d0-b9f4-137db4741fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:19.301073 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a814008-c6f4-44d0-b9f4-137db4741fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:19.301555 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a814008-c6f4-44d0-b9f4-137db4741fe3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:19.312627 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a814008-c6f4-44d0-b9f4-137db4741fe3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:19.314091 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a814008-c6f4-44d0-b9f4-137db4741fe3 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:19.333232 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-9a814008-c6f4-44d0-b9f4-137db4741fe3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:16:19.333232 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:16:19.333232 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:16:19.333232 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:16:19.333232 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:16:19.333232 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:16:19.333232 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:16:19.333232 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:16:19.333232 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:16:19.333232 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:16:19.333232 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:16:19.333232 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:16:19.333232 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:16:19.333232 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:16:19.333232 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:16:19.333232 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:16:19.334289 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:16:19.334289 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:16:19.334289 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:16:19.334289 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:16:19.334289 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:16:19.334289 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:16:19.334289 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:16:19.334289 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:16:19.334289 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:16:19.334289 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:16:19.334289 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:16:19.334289 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:16:19.334289 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:16:19.334289 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:16:19.334289 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:16:19.334289 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:16:19.334289 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:16:19.335770 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:16:19.335770 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:16:19.335770 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:16:19.335770 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 605/1215] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:16:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:16:19.340334 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5696dd7-6bb8-4026-aa0e-4a74f9e4bbac None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:19.405828 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:19.452320 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5696dd7-6bb8-4026-aa0e-4a74f9e4bbac None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:19.492031 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4231e84a-8458-44fa-8250-77287e37e87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:19.493029 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231e84a-8458-44fa-8250-77287e37e87f None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:19.493524 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231e84a-8458-44fa-8250-77287e37e87f None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:19.494002 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231e84a-8458-44fa-8250-77287e37e87f None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:19.494960 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4231e84a-8458-44fa-8250-77287e37e87f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:16:19.495462 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4231e84a-8458-44fa-8250-77287e37e87f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:19.497081 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4231e84a-8458-44fa-8250-77287e37e87f None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:19.508674 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 611/1216] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:16:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:16:19.514648 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a814008-c6f4-44d0-b9f4-137db4741fe3 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:19.601389 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:19.635219 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a814008-c6f4-44d0-b9f4-137db4741fe3 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:19.690872 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124d3c0-ab1b-4792-8b89-85de10711e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:19.692264 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124d3c0-ab1b-4792-8b89-85de10711e64 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:19.692657 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124d3c0-ab1b-4792-8b89-85de10711e64 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:19.693196 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124d3c0-ab1b-4792-8b89-85de10711e64 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:19.704974 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2124d3c0-ab1b-4792-8b89-85de10711e64 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:19.708009 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2124d3c0-ab1b-4792-8b89-85de10711e64 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:19.729755 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-2124d3c0-ab1b-4792-8b89-85de10711e64 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:16:19.729755 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:16:19.729755 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:16:19.729755 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:16:19.729755 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:16:19.729755 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:16:19.729755 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:16:19.729755 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:16:19.729755 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:16:19.729755 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:16:19.729755 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:16:19.729755 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:16:19.729755 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:16:19.729755 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:16:19.729755 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:16:19.731826 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:16:19.731826 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:16:19.731826 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:16:19.731826 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:16:19.731826 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:16:19.731826 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:16:19.731826 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:16:19.731826 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:16:19.731826 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:16:19.731826 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:16:19.731826 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:16:19.731826 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:16:19.731826 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:16:19.731826 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:16:19.731826 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:16:19.731826 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:16:19.731826 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:16:19.733821 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:16:19.733821 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:16:19.733821 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:16:19.733821 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:16:19.733821 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:16:19.733821 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:16:19.733821 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:16:19.733821 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:16:19.733821 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 606/1217] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:16:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:16:19.740316 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4231e84a-8458-44fa-8250-77287e37e87f None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:19.863502 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4231e84a-8458-44fa-8250-77287e37e87f None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:19.889493 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:19.903425 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d90078-87e2-45dc-834a-8e2ab888f84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:19.904192 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d90078-87e2-45dc-834a-8e2ab888f84d None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:19.904574 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d90078-87e2-45dc-834a-8e2ab888f84d None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:19.904890 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d90078-87e2-45dc-834a-8e2ab888f84d None admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:19.905691 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d90078-87e2-45dc-834a-8e2ab888f84d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:16:19.905995 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d90078-87e2-45dc-834a-8e2ab888f84d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:19.907456 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d90078-87e2-45dc-834a-8e2ab888f84d None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:19.921460 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 612/1218] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 14:16:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:16:19.928328 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124d3c0-ab1b-4792-8b89-85de10711e64 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:20.037316 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124d3c0-ab1b-4792-8b89-85de10711e64 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:20.074099 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:20.080968 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d9f62-2dfe-4a09-b80d-7f88ef2764d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:20.081651 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d9f62-2dfe-4a09-b80d-7f88ef2764d5 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:20.081941 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d9f62-2dfe-4a09-b80d-7f88ef2764d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:20.082276 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d9f62-2dfe-4a09-b80d-7f88ef2764d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:20.083085 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33d9f62-2dfe-4a09-b80d-7f88ef2764d5 None admin] RBAC: Adding query filter params (role.id=be958e9954f04268a9f012c775fc17e5, scope.project.id=36a3ae7bab25471fb989136150b6438c, user.id=18fe384ecd454fb298dcc788f41dd44b) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:16:20.083398 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33d9f62-2dfe-4a09-b80d-7f88ef2764d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:20.084861 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33d9f62-2dfe-4a09-b80d-7f88ef2764d5 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:20.090447 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 607/1219] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:16:19 2022] GET /identity/v3/role_assignments?user.id=18fe384ecd454fb298dcc788f41dd44b&scope.project.id=36a3ae7bab25471fb989136150b6438c&role.id=be958e9954f04268a9f012c775fc17e5 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:16:22.345328 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ffccf-66b1-4f3b-a236-c81ec660f526 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:22.345935 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ffccf-66b1-4f3b-a236-c81ec660f526 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:22.345935 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ffccf-66b1-4f3b-a236-c81ec660f526 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:22.347293 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 613/1220] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:16:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:16:22.353229 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2246a0d-3663-470e-b92d-d7fe9931fe5c None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:22.353694 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2246a0d-3663-470e-b92d-d7fe9931fe5c None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:22.353809 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2246a0d-3663-470e-b92d-d7fe9931fe5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:22.375800 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-d2246a0d-3663-470e-b92d-d7fe9931fe5c None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:16:22.376995 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-d2246a0d-3663-470e-b92d-d7fe9931fe5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:16:22.449150 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 608/1221] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:16:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:16:22.739769 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ffccf-66b1-4f3b-a236-c81ec660f526 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:22.853036 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ffccf-66b1-4f3b-a236-c81ec660f526 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:22.889644 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d81677-e1de-4d81-a420-a329524fb6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:22.890278 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d81677-e1de-4d81-a420-a329524fb6a0 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:22.890557 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d81677-e1de-4d81-a420-a329524fb6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:22.890844 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d81677-e1de-4d81-a420-a329524fb6a0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:22.901197 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d81677-e1de-4d81-a420-a329524fb6a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:16:22.905964 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-21d81677-e1de-4d81-a420-a329524fb6a0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:16:22.905964 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:16:22.905964 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:16:22.905964 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:16:22.905964 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:16:22.905964 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:16:22.905964 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:16:22.905964 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:16:22.905964 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:16:22.905964 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:16:22.905964 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:16:22.905964 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:16:22.905964 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:16:22.905964 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:16:22.905964 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:16:22.905964 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:16:22.907141 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:16:22.907141 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:16:22.907141 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:16:22.907141 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:16:22.907141 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:16:22.907141 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:16:22.907141 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:16:22.907141 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:16:22.907141 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:16:22.907141 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:16:22.907141 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:16:22.907141 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:16:22.907141 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:16:22.907141 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:16:22.907141 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:16:22.907141 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:16:22.908621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:16:22.908621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:16:22.908621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:16:22.908621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:16:22.908621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:16:22.908621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:16:22.908621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:16:22.908621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:16:22.908621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:16:22.908621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:16:22.908621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:16:22.908621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:16:22.908621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 614/1222] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 14:16:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:16:22.914364 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2246a0d-3663-470e-b92d-d7fe9931fe5c None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:23.021966 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2246a0d-3663-470e-b92d-d7fe9931fe5c None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:23.049601 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:23.062016 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42162307-1c70-430d-aa42-bd76ce17b438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:23.062632 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42162307-1c70-430d-aa42-bd76ce17b438 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:23.062937 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42162307-1c70-430d-aa42-bd76ce17b438 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:23.063205 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42162307-1c70-430d-aa42-bd76ce17b438 None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:23.063895 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42162307-1c70-430d-aa42-bd76ce17b438 None admin] RBAC: Adding query filter params (name=service) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:16:23.064312 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42162307-1c70-430d-aa42-bd76ce17b438 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:23.065587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42162307-1c70-430d-aa42-bd76ce17b438 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:23.075141 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 609/1223] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 14:16:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:16:23.081061 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d81677-e1de-4d81-a420-a329524fb6a0 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:23.129789 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:23.194877 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d81677-e1de-4d81-a420-a329524fb6a0 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:23.230604 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f569b45c-8e43-4e96-aebd-013b6ea6d890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:23.231342 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569b45c-8e43-4e96-aebd-013b6ea6d890 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:23.231666 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569b45c-8e43-4e96-aebd-013b6ea6d890 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:23.231938 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569b45c-8e43-4e96-aebd-013b6ea6d890 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:23.243728 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f569b45c-8e43-4e96-aebd-013b6ea6d890 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:23.245304 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f569b45c-8e43-4e96-aebd-013b6ea6d890 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:23.262451 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-f569b45c-8e43-4e96-aebd-013b6ea6d890 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:16:23.262451 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:16:23.262451 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:16:23.262451 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:16:23.262451 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:16:23.262451 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:16:23.262451 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:16:23.262451 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:16:23.262451 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:16:23.262451 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:16:23.262451 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:16:23.262451 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:16:23.262451 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:16:23.262451 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:16:23.262451 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:16:23.262451 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:16:23.263755 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:16:23.263755 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:16:23.263755 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:16:23.263755 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:16:23.263755 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:16:23.263755 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:16:23.263755 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:16:23.263755 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:16:23.263755 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:16:23.263755 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:16:23.263755 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:16:23.263755 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:16:23.263755 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:16:23.263755 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:16:23.263755 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:16:23.263755 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:16:23.263755 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:16:23.264883 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:16:23.264883 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:16:23.264883 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:16:23.264883 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 615/1224] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:16:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:16:23.271220 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42162307-1c70-430d-aa42-bd76ce17b438 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:23.375312 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42162307-1c70-430d-aa42-bd76ce17b438 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:23.411702 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337fda19-5508-4231-8fe2-7e5e29e08c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:23.412310 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337fda19-5508-4231-8fe2-7e5e29e08c0a None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:23.412585 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337fda19-5508-4231-8fe2-7e5e29e08c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:23.412852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337fda19-5508-4231-8fe2-7e5e29e08c0a None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:23.413488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337fda19-5508-4231-8fe2-7e5e29e08c0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:16:23.413784 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337fda19-5508-4231-8fe2-7e5e29e08c0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:23.415185 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337fda19-5508-4231-8fe2-7e5e29e08c0a None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:23.430172 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 610/1225] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:16:23.435894 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f569b45c-8e43-4e96-aebd-013b6ea6d890 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:23.535230 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:23.543406 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f569b45c-8e43-4e96-aebd-013b6ea6d890 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:23.589576 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b112e134-c1e8-41c1-b524-8f2b7d446805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:23.590354 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b112e134-c1e8-41c1-b524-8f2b7d446805 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:23.590632 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b112e134-c1e8-41c1-b524-8f2b7d446805 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:23.590936 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b112e134-c1e8-41c1-b524-8f2b7d446805 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:23.600141 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b112e134-c1e8-41c1-b524-8f2b7d446805 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:23.601091 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b112e134-c1e8-41c1-b524-8f2b7d446805 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:23.614575 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-b112e134-c1e8-41c1-b524-8f2b7d446805 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 24 14:16:23.614575 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:16:23.614575 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:16:23.614575 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:16:23.614575 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:16:23.614575 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:16:23.614575 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:16:23.614575 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:16:23.614575 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:16:23.614575 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:16:23.614575 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:16:23.614575 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:16:23.614575 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:16:23.614575 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:16:23.614575 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:16:23.614575 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:16:23.620984 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:16:23.620984 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:16:23.620984 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:16:23.620984 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:16:23.620984 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:16:23.620984 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:16:23.620984 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:16:23.620984 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:16:23.620984 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:16:23.620984 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:16:23.620984 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:16:23.620984 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:16:23.620984 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:16:23.620984 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:16:23.620984 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:16:23.620984 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:16:23.620984 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:16:23.620984 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:16:23.622233 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:16:23.622233 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:16:23.622233 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:16:23.622233 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:16:23.622233 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:16:23.622233 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:16:23.622233 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:16:23.622233 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:16:23.622233 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 24 14:16:23.622233 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:16:23.622233 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 616/1226] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:16:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:16:23.623088 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337fda19-5508-4231-8fe2-7e5e29e08c0a None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:23.674073 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:23.735666 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337fda19-5508-4231-8fe2-7e5e29e08c0a None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:23.787097 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113fcf96-2aa7-424e-9ade-d3d22d03c892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:23.787934 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113fcf96-2aa7-424e-9ade-d3d22d03c892 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:23.788215 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113fcf96-2aa7-424e-9ade-d3d22d03c892 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:23.788678 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113fcf96-2aa7-424e-9ade-d3d22d03c892 None admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:23.789772 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113fcf96-2aa7-424e-9ade-d3d22d03c892 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:16:23.790076 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113fcf96-2aa7-424e-9ade-d3d22d03c892 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:23.791529 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113fcf96-2aa7-424e-9ade-d3d22d03c892 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:23.816789 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 611/1227] 10.222.0.52 () {56 vars in 1143 bytes} [Sat Sep 24 14:16:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:16:23.822793 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b112e134-c1e8-41c1-b524-8f2b7d446805 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:23.935785 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b112e134-c1e8-41c1-b524-8f2b7d446805 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:23.977234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3ae562-5f04-48f4-bab1-3cbe1dabab21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:23.978210 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ae562-5f04-48f4-bab1-3cbe1dabab21 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:23.978490 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ae562-5f04-48f4-bab1-3cbe1dabab21 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:23.978767 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ae562-5f04-48f4-bab1-3cbe1dabab21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:23.990130 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3ae562-5f04-48f4-bab1-3cbe1dabab21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:23.991751 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3ae562-5f04-48f4-bab1-3cbe1dabab21 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:24.007128 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-5d3ae562-5f04-48f4-bab1-3cbe1dabab21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:16:24.007128 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:16:24.007128 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:16:24.007128 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:16:24.007128 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:16:24.007128 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:16:24.007128 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:16:24.007128 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:16:24.007128 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:16:24.007128 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:16:24.007128 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:16:24.007128 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:16:24.007128 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:16:24.007128 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:16:24.007128 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:16:24.007128 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:16:24.012828 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:16:24.012828 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:16:24.012828 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:16:24.012828 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:16:24.012828 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:16:24.012828 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:16:24.012828 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:16:24.012828 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:16:24.012828 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:16:24.012828 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:16:24.012828 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:16:24.012828 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:16:24.012828 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:16:24.012828 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:16:24.012828 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:16:24.012828 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:16:24.012828 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:16:24.013973 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:16:24.013973 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:16:24.013973 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:16:24.013973 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 617/1228] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:16:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:16:24.014254 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113fcf96-2aa7-424e-9ade-d3d22d03c892 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:24.120529 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113fcf96-2aa7-424e-9ade-d3d22d03c892 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:24.123101 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:24.159332 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e679c6e9-d564-45ca-aaa9-c71f4bd3e8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:24.160576 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679c6e9-d564-45ca-aaa9-c71f4bd3e8d9 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:24.160880 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679c6e9-d564-45ca-aaa9-c71f4bd3e8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:24.161173 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679c6e9-d564-45ca-aaa9-c71f4bd3e8d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:24.161865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e679c6e9-d564-45ca-aaa9-c71f4bd3e8d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:16:24.162191 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e679c6e9-d564-45ca-aaa9-c71f4bd3e8d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:24.163762 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e679c6e9-d564-45ca-aaa9-c71f4bd3e8d9 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:24.180471 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 612/1229] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:16:24.181620 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3ae562-5f04-48f4-bab1-3cbe1dabab21 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:24.191821 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:24.292614 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3ae562-5f04-48f4-bab1-3cbe1dabab21 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:24.329861 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d42825-3bce-4525-8df7-616488632c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:24.330510 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d42825-3bce-4525-8df7-616488632c2d None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:24.330793 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d42825-3bce-4525-8df7-616488632c2d None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:24.331106 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d42825-3bce-4525-8df7-616488632c2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:24.338247 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d42825-3bce-4525-8df7-616488632c2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:24.339717 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d42825-3bce-4525-8df7-616488632c2d None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:24.352449 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-12d42825-3bce-4525-8df7-616488632c2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:16:24.352449 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:16:24.352449 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:16:24.352449 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:16:24.352449 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:16:24.352449 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:16:24.352449 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:16:24.352449 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:16:24.352449 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:16:24.352449 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:16:24.352449 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:16:24.352449 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:16:24.352449 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:16:24.352449 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:16:24.352449 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:16:24.353447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:16:24.353447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:16:24.353447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:16:24.353447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:16:24.353447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:16:24.353447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:16:24.353447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:16:24.353447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:16:24.353447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:16:24.353447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:16:24.353447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:16:24.353447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:16:24.353447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:16:24.353447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:16:24.353447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:16:24.353447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:16:24.353447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:16:24.354360 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:16:24.354360 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:16:24.354360 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:16:24.354360 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:16:24.354360 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:16:24.354360 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:16:24.354360 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:16:24.354360 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:16:24.354360 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 618/1230] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:16:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:16:24.359833 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e679c6e9-d564-45ca-aaa9-c71f4bd3e8d9 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:24.466298 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e679c6e9-d564-45ca-aaa9-c71f4bd3e8d9 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:24.502669 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fccf67-6c17-4c89-9f41-0106526ed76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:24.503354 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fccf67-6c17-4c89-9f41-0106526ed76c None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:24.503678 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fccf67-6c17-4c89-9f41-0106526ed76c None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:24.504013 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fccf67-6c17-4c89-9f41-0106526ed76c None admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:24.504802 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fccf67-6c17-4c89-9f41-0106526ed76c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:16:24.505132 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fccf67-6c17-4c89-9f41-0106526ed76c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:24.506658 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fccf67-6c17-4c89-9f41-0106526ed76c None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:24.518506 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 613/1231] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 14:16:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:16:24.525060 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d42825-3bce-4525-8df7-616488632c2d None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:24.603435 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:24.641350 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d42825-3bce-4525-8df7-616488632c2d None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:24.683596 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10140c6a-301a-4b8f-a526-8709ead8ebc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:24.684252 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10140c6a-301a-4b8f-a526-8709ead8ebc0 None admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:24.684548 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10140c6a-301a-4b8f-a526-8709ead8ebc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:24.684817 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10140c6a-301a-4b8f-a526-8709ead8ebc0 None admin] PATH_INFO: `/v3/projects/36a3ae7bab25471fb989136150b6438c/users/18fe384ecd454fb298dcc788f41dd44b/roles/be958e9954f04268a9f012c775fc17e5` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:24.699229 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10140c6a-301a-4b8f-a526-8709ead8ebc0 None admin] RBAC: Authorizing `identity:create_grant(project_id=36a3ae7bab25471fb989136150b6438c, user_id=18fe384ecd454fb298dcc788f41dd44b, role_id=be958e9954f04268a9f012c775fc17e5)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:24.700674 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10140c6a-301a-4b8f-a526-8709ead8ebc0 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:24.714333 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 619/1232] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 14:16:24 2022] PUT /identity/v3/projects/36a3ae7bab25471fb989136150b6438c/users/18fe384ecd454fb298dcc788f41dd44b/roles/be958e9954f04268a9f012c775fc17e5 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:16:26.928908 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666431e-9e0f-4028-91e4-d389b559d18f None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:26.929507 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666431e-9e0f-4028-91e4-d389b559d18f None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:26.929507 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666431e-9e0f-4028-91e4-d389b559d18f None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:26.930803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 614/1233] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:16:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:16:26.936360 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a1e34-2fe7-4ae5-813a-7917bc05ae81 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:26.936883 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a1e34-2fe7-4ae5-813a-7917bc05ae81 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:26.936883 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a1e34-2fe7-4ae5-813a-7917bc05ae81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:26.955851 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-084a1e34-2fe7-4ae5-813a-7917bc05ae81 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:16:26.957449 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-084a1e34-2fe7-4ae5-813a-7917bc05ae81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:16:27.028523 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 620/1234] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:16:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:16:27.307487 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b666431e-9e0f-4028-91e4-d389b559d18f None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:27.333351 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:27.416374 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b666431e-9e0f-4028-91e4-d389b559d18f None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:27.459775 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ab3252-7e08-4f00-88d1-df79dd35a19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:27.460504 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab3252-7e08-4f00-88d1-df79dd35a19c None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:27.460794 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab3252-7e08-4f00-88d1-df79dd35a19c None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:27.461064 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab3252-7e08-4f00-88d1-df79dd35a19c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:27.473118 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ab3252-7e08-4f00-88d1-df79dd35a19c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:16:27.478535 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-62ab3252-7e08-4f00-88d1-df79dd35a19c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:16:27.478535 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:16:27.478535 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:16:27.478535 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:16:27.478535 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:16:27.478535 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:16:27.478535 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:16:27.478535 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:16:27.478535 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:16:27.478535 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:16:27.478535 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:16:27.478535 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:16:27.478535 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:16:27.478535 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:16:27.478535 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:16:27.478535 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:16:27.479817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:16:27.479817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:16:27.479817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:16:27.479817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:16:27.479817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:16:27.479817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:16:27.479817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:16:27.479817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:16:27.479817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:16:27.479817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:16:27.479817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:16:27.479817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:16:27.479817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:16:27.479817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:16:27.479817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:16:27.479817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:16:27.480914 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:16:27.480914 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:16:27.480914 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:16:27.480914 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:16:27.480914 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:16:27.480914 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:16:27.480914 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:16:27.480914 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:16:27.480914 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:16:27.480914 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:16:27.480914 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 14:16:27.480914 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:16:27.480914 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 615/1235] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 14:16:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:16:27.486051 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084a1e34-2fe7-4ae5-813a-7917bc05ae81 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:27.541117 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:27.607701 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084a1e34-2fe7-4ae5-813a-7917bc05ae81 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:27.648059 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5422ac90-838e-49e5-93bb-1104a0e52530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:27.648831 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422ac90-838e-49e5-93bb-1104a0e52530 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:27.649132 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422ac90-838e-49e5-93bb-1104a0e52530 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:27.649499 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422ac90-838e-49e5-93bb-1104a0e52530 None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:27.650225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5422ac90-838e-49e5-93bb-1104a0e52530 None admin] RBAC: Adding query filter params (name=service) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:16:27.650478 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5422ac90-838e-49e5-93bb-1104a0e52530 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:27.652391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5422ac90-838e-49e5-93bb-1104a0e52530 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:27.663285 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 621/1236] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 14:16:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:16:27.670887 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ab3252-7e08-4f00-88d1-df79dd35a19c None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:27.767434 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:27.787228 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ab3252-7e08-4f00-88d1-df79dd35a19c None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:27.840392 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd04383-46f6-4321-9145-580f7a1eb0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:27.841038 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd04383-46f6-4321-9145-580f7a1eb0fe None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:27.841318 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd04383-46f6-4321-9145-580f7a1eb0fe None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:27.841589 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd04383-46f6-4321-9145-580f7a1eb0fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:27.853564 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd04383-46f6-4321-9145-580f7a1eb0fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:27.855034 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd04383-46f6-4321-9145-580f7a1eb0fe None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:27.871488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-6fd04383-46f6-4321-9145-580f7a1eb0fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:16:27.871488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:16:27.871488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:16:27.871488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:16:27.871488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:16:27.871488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:16:27.871488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:16:27.871488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:16:27.871488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:16:27.871488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:16:27.871488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:16:27.871488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:16:27.871488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:16:27.871488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:16:27.871488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:16:27.871488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:16:27.872601 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:16:27.872601 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:16:27.872601 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:16:27.872601 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:16:27.872601 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:16:27.872601 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:16:27.872601 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:16:27.872601 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:16:27.872601 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:16:27.872601 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:16:27.872601 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:16:27.872601 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:16:27.872601 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:16:27.872601 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:16:27.872601 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:16:27.872601 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:16:27.872601 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:16:27.873723 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:16:27.873723 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:16:27.873723 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:16:27.873723 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 616/1237] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:16:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:16:27.878947 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5422ac90-838e-49e5-93bb-1104a0e52530 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:27.981386 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5422ac90-838e-49e5-93bb-1104a0e52530 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:27.990990 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:28.021923 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5295d558-1863-4d71-8695-65edd1ba575c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:28.022543 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295d558-1863-4d71-8695-65edd1ba575c None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:28.022843 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295d558-1863-4d71-8695-65edd1ba575c None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:28.023152 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295d558-1863-4d71-8695-65edd1ba575c None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:28.023828 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5295d558-1863-4d71-8695-65edd1ba575c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:16:28.024152 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5295d558-1863-4d71-8695-65edd1ba575c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:28.025602 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5295d558-1863-4d71-8695-65edd1ba575c None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:28.039148 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 622/1238] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:16:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:16:28.044850 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd04383-46f6-4321-9145-580f7a1eb0fe None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:28.152500 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd04383-46f6-4321-9145-580f7a1eb0fe None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:28.191036 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527729f0-df29-4a31-a076-2be915ca9de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:28.192043 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527729f0-df29-4a31-a076-2be915ca9de0 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:28.192506 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527729f0-df29-4a31-a076-2be915ca9de0 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:28.192952 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527729f0-df29-4a31-a076-2be915ca9de0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:28.197093 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:28.203403 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527729f0-df29-4a31-a076-2be915ca9de0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:28.205001 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527729f0-df29-4a31-a076-2be915ca9de0 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:28.219162 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-527729f0-df29-4a31-a076-2be915ca9de0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 24 14:16:28.219162 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:16:28.219162 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:16:28.219162 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:16:28.219162 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:16:28.219162 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:16:28.219162 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:16:28.219162 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:16:28.219162 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:16:28.219162 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:16:28.219162 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:16:28.219162 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:16:28.219162 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:16:28.219162 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:16:28.219162 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:16:28.219162 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:16:28.220353 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:16:28.220353 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:16:28.220353 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:16:28.220353 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:16:28.220353 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:16:28.220353 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:16:28.220353 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:16:28.220353 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:16:28.220353 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:16:28.220353 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:16:28.220353 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:16:28.220353 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:16:28.220353 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:16:28.220353 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:16:28.220353 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:16:28.220353 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:16:28.220353 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:16:28.220353 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:16:28.221540 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:16:28.221540 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:16:28.221540 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:16:28.221540 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:16:28.221540 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:16:28.221540 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:16:28.221540 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:16:28.221540 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:16:28.221540 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 24 14:16:28.221540 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:16:28.221540 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 617/1239] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:16:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:16:28.229143 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5295d558-1863-4d71-8695-65edd1ba575c None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:28.342786 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5295d558-1863-4d71-8695-65edd1ba575c None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:28.382207 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb71cd5-5a09-44f2-852f-aae1993845be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:28.382857 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb71cd5-5a09-44f2-852f-aae1993845be None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:28.383131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb71cd5-5a09-44f2-852f-aae1993845be None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:28.383406 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb71cd5-5a09-44f2-852f-aae1993845be None admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:28.384191 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb71cd5-5a09-44f2-852f-aae1993845be None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:16:28.384515 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb71cd5-5a09-44f2-852f-aae1993845be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:28.386032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb71cd5-5a09-44f2-852f-aae1993845be None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:28.406258 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 623/1240] 10.222.0.52 () {56 vars in 1143 bytes} [Sat Sep 24 14:16:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:16:28.412568 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527729f0-df29-4a31-a076-2be915ca9de0 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:28.522914 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527729f0-df29-4a31-a076-2be915ca9de0 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:28.567867 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabe9d54-6c80-413d-a140-552bc9564d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:28.568537 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe9d54-6c80-413d-a140-552bc9564d6e None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:28.568875 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe9d54-6c80-413d-a140-552bc9564d6e None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:28.569195 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe9d54-6c80-413d-a140-552bc9564d6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:28.651222 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:28.659108 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabe9d54-6c80-413d-a140-552bc9564d6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:28.661316 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabe9d54-6c80-413d-a140-552bc9564d6e None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:28.780637 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 618/1241] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:16:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 14:16:28.787487 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb71cd5-5a09-44f2-852f-aae1993845be None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:28.816457 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:28.916959 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb71cd5-5a09-44f2-852f-aae1993845be None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:28.966912 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776123d-e577-4cbc-b961-9dc843392b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:28.967924 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776123d-e577-4cbc-b961-9dc843392b1a None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:28.968360 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776123d-e577-4cbc-b961-9dc843392b1a None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:28.968808 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776123d-e577-4cbc-b961-9dc843392b1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:28.984577 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3776123d-e577-4cbc-b961-9dc843392b1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:28.986776 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3776123d-e577-4cbc-b961-9dc843392b1a None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:29.009803 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-3776123d-e577-4cbc-b961-9dc843392b1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:16:29.009803 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:16:29.009803 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:16:29.009803 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:16:29.009803 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:16:29.009803 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:16:29.009803 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:16:29.009803 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:16:29.009803 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:16:29.009803 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:16:29.009803 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:16:29.009803 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:16:29.009803 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:16:29.009803 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:16:29.009803 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:16:29.009803 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:16:29.011089 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:16:29.011089 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:16:29.011089 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:16:29.011089 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:16:29.011089 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:16:29.011089 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:16:29.011089 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:16:29.011089 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:16:29.011089 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:16:29.011089 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:16:29.011089 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:16:29.011089 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:16:29.011089 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:16:29.011089 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:16:29.011089 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:16:29.011089 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:16:29.011089 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:16:29.012215 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:16:29.012215 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:16:29.012215 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:16:29.012215 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 624/1242] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:16:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:16:29.017516 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabe9d54-6c80-413d-a140-552bc9564d6e None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:29.133784 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:29.141536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabe9d54-6c80-413d-a140-552bc9564d6e None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:29.182783 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931423b4-7a0c-40a1-b3f4-205dcd0f8827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:29.183319 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931423b4-7a0c-40a1-b3f4-205dcd0f8827 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:29.183623 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931423b4-7a0c-40a1-b3f4-205dcd0f8827 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:29.183897 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931423b4-7a0c-40a1-b3f4-205dcd0f8827 None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:29.184551 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931423b4-7a0c-40a1-b3f4-205dcd0f8827 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:16:29.184849 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931423b4-7a0c-40a1-b3f4-205dcd0f8827 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:29.186319 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931423b4-7a0c-40a1-b3f4-205dcd0f8827 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:29.198674 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 619/1243] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:16:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:16:29.204801 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776123d-e577-4cbc-b961-9dc843392b1a None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:29.300708 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:29.324806 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776123d-e577-4cbc-b961-9dc843392b1a None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:29.370033 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87df9e76-a548-45d9-bcd4-2e296065c8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:29.370745 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df9e76-a548-45d9-bcd4-2e296065c8c9 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:29.371169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df9e76-a548-45d9-bcd4-2e296065c8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:29.371465 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df9e76-a548-45d9-bcd4-2e296065c8c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:29.379452 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87df9e76-a548-45d9-bcd4-2e296065c8c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:29.381125 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87df9e76-a548-45d9-bcd4-2e296065c8c9 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:29.394381 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-87df9e76-a548-45d9-bcd4-2e296065c8c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:16:29.394381 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:16:29.394381 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:16:29.394381 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:16:29.394381 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:16:29.394381 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:16:29.394381 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:16:29.394381 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:16:29.394381 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:16:29.394381 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:16:29.394381 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:16:29.394381 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:16:29.394381 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:16:29.394381 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:16:29.394381 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:16:29.395613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:16:29.395613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:16:29.395613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:16:29.395613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:16:29.395613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:16:29.395613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:16:29.395613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:16:29.395613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:16:29.395613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:16:29.395613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:16:29.395613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:16:29.395613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:16:29.395613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:16:29.395613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:16:29.395613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:16:29.395613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:16:29.395613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:16:29.396724 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:16:29.396724 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:16:29.396724 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:16:29.396724 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:16:29.396724 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:16:29.396724 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:16:29.396724 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:16:29.396724 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:16:29.396724 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 625/1244] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:16:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:16:29.401697 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931423b4-7a0c-40a1-b3f4-205dcd0f8827 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:29.524636 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931423b4-7a0c-40a1-b3f4-205dcd0f8827 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:29.563533 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c1a46-5c10-4751-ba72-e2522fe3eb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:29.564232 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c1a46-5c10-4751-ba72-e2522fe3eb29 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:29.564519 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c1a46-5c10-4751-ba72-e2522fe3eb29 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:29.564787 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c1a46-5c10-4751-ba72-e2522fe3eb29 None admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:29.565692 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c1a46-5c10-4751-ba72-e2522fe3eb29 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:16:29.565999 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c1a46-5c10-4751-ba72-e2522fe3eb29 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:29.567563 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c1a46-5c10-4751-ba72-e2522fe3eb29 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:29.581189 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 620/1245] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 14:16:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:16:29.588268 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87df9e76-a548-45d9-bcd4-2e296065c8c9 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:29.711270 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87df9e76-a548-45d9-bcd4-2e296065c8c9 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:29.761288 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b8c0a-4e7e-4565-871f-b6e12d20de0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:29.761893 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b8c0a-4e7e-4565-871f-b6e12d20de0a None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:29.762176 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b8c0a-4e7e-4565-871f-b6e12d20de0a None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:29.762432 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b8c0a-4e7e-4565-871f-b6e12d20de0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:29.763179 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796b8c0a-4e7e-4565-871f-b6e12d20de0a None admin] RBAC: Adding query filter params (role.id=be958e9954f04268a9f012c775fc17e5, scope.project.id=36a3ae7bab25471fb989136150b6438c, user.id=18fe384ecd454fb298dcc788f41dd44b) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:16:29.763521 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796b8c0a-4e7e-4565-871f-b6e12d20de0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:29.765129 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796b8c0a-4e7e-4565-871f-b6e12d20de0a None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:29.773884 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 626/1246] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:16:29 2022] GET /identity/v3/role_assignments?user.id=18fe384ecd454fb298dcc788f41dd44b&scope.project.id=36a3ae7bab25471fb989136150b6438c&role.id=be958e9954f04268a9f012c775fc17e5 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:16:32.313053 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f3e9c-8021-459d-8951-6975bc8207bd None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:32.313630 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f3e9c-8021-459d-8951-6975bc8207bd None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:32.313630 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f3e9c-8021-459d-8951-6975bc8207bd None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:32.315329 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 621/1247] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:16:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:16:32.321830 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84ac21-82ec-4132-b757-5b5db49bdcec None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:32.322260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84ac21-82ec-4132-b757-5b5db49bdcec None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:32.322505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84ac21-82ec-4132-b757-5b5db49bdcec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:32.329605 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:32.344218 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-4a84ac21-82ec-4132-b757-5b5db49bdcec None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:16:32.345835 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-4a84ac21-82ec-4132-b757-5b5db49bdcec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:16:32.429640 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 627/1248] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:16:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:16:32.715159 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f3e9c-8021-459d-8951-6975bc8207bd None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:32.737396 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:32.845332 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f3e9c-8021-459d-8951-6975bc8207bd None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:32.883191 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec78a9-9d89-4e7f-9b2b-8922da57241f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:32.883835 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec78a9-9d89-4e7f-9b2b-8922da57241f None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:32.884108 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec78a9-9d89-4e7f-9b2b-8922da57241f None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:32.884380 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec78a9-9d89-4e7f-9b2b-8922da57241f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:32.894858 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feec78a9-9d89-4e7f-9b2b-8922da57241f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:16:32.899859 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-feec78a9-9d89-4e7f-9b2b-8922da57241f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:16:32.899859 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:16:32.899859 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:16:32.899859 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:16:32.899859 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:16:32.899859 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:16:32.899859 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:16:32.899859 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:16:32.899859 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:16:32.899859 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:16:32.899859 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:16:32.899859 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:16:32.899859 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:16:32.899859 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:16:32.899859 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:16:32.899859 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:16:32.902618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:16:32.902618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:16:32.902618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:16:32.902618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:16:32.902618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:16:32.902618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:16:32.902618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:16:32.902618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:16:32.902618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:16:32.902618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:16:32.902618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:16:32.902618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:16:32.902618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:16:32.902618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:16:32.902618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:16:32.902618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:16:32.903514 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:16:32.903514 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:16:32.903514 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:16:32.903514 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:16:32.903514 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:16:32.903514 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:16:32.903514 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:16:32.903514 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:16:32.903514 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:16:32.903514 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:16:32.903514 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:16:32.903514 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:16:32.903514 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 622/1249] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 14:16:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:16:32.906897 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a84ac21-82ec-4132-b757-5b5db49bdcec None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:33.017779 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a84ac21-82ec-4132-b757-5b5db49bdcec None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:33.052780 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:33.059230 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b0f022-93bc-4767-976f-45b6bfb0ee7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:33.059782 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0f022-93bc-4767-976f-45b6bfb0ee7c None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:33.060207 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0f022-93bc-4767-976f-45b6bfb0ee7c None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:33.060511 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0f022-93bc-4767-976f-45b6bfb0ee7c None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:33.061208 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b0f022-93bc-4767-976f-45b6bfb0ee7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:16:33.061512 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b0f022-93bc-4767-976f-45b6bfb0ee7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:33.063065 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b0f022-93bc-4767-976f-45b6bfb0ee7c None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:33.071846 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 628/1250] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:16:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:16:33.078465 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec78a9-9d89-4e7f-9b2b-8922da57241f None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:33.169361 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:33.194501 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec78a9-9d89-4e7f-9b2b-8922da57241f None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:33.231942 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c422c5-0bd2-4678-98f0-5c050a95cb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:33.232716 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c422c5-0bd2-4678-98f0-5c050a95cb37 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:33.232995 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c422c5-0bd2-4678-98f0-5c050a95cb37 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:33.233264 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c422c5-0bd2-4678-98f0-5c050a95cb37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:33.244844 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c422c5-0bd2-4678-98f0-5c050a95cb37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:33.246282 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c422c5-0bd2-4678-98f0-5c050a95cb37 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:33.263086 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-30c422c5-0bd2-4678-98f0-5c050a95cb37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:16:33.263086 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:16:33.263086 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:16:33.263086 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:16:33.263086 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:16:33.263086 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:16:33.263086 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:16:33.263086 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:16:33.263086 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:16:33.263086 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:16:33.263086 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:16:33.263086 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:16:33.263086 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:16:33.263086 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:16:33.263086 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:16:33.263086 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:16:33.264287 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:16:33.264287 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:16:33.264287 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:16:33.264287 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:16:33.264287 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:16:33.264287 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:16:33.264287 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:16:33.264287 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:16:33.264287 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:16:33.264287 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:16:33.264287 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:16:33.264287 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:16:33.264287 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:16:33.264287 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:16:33.264287 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:16:33.264287 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:16:33.264287 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:16:33.265405 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:16:33.265405 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:16:33.265405 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:16:33.269663 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 623/1251] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:16:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:16:33.270700 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b0f022-93bc-4767-976f-45b6bfb0ee7c None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:33.386376 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b0f022-93bc-4767-976f-45b6bfb0ee7c None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:33.424497 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05a3c4-bbca-4ae2-bb97-9aa6ad75bd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:33.425110 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05a3c4-bbca-4ae2-bb97-9aa6ad75bd77 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:33.425386 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05a3c4-bbca-4ae2-bb97-9aa6ad75bd77 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:33.425657 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05a3c4-bbca-4ae2-bb97-9aa6ad75bd77 None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:33.426298 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d05a3c4-bbca-4ae2-bb97-9aa6ad75bd77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:16:33.426599 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d05a3c4-bbca-4ae2-bb97-9aa6ad75bd77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:33.428006 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d05a3c4-bbca-4ae2-bb97-9aa6ad75bd77 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:33.441369 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 629/1252] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:16:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:16:33.446891 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c422c5-0bd2-4678-98f0-5c050a95cb37 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:33.578195 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c422c5-0bd2-4678-98f0-5c050a95cb37 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:33.613441 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:33.622849 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a6c8c9-08a8-43db-b2a5-5ca6d48c0f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:33.623536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a6c8c9-08a8-43db-b2a5-5ca6d48c0f9f None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:33.623812 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a6c8c9-08a8-43db-b2a5-5ca6d48c0f9f None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:33.624094 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a6c8c9-08a8-43db-b2a5-5ca6d48c0f9f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:33.633162 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a6c8c9-08a8-43db-b2a5-5ca6d48c0f9f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:33.634655 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a6c8c9-08a8-43db-b2a5-5ca6d48c0f9f None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:33.646905 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-97a6c8c9-08a8-43db-b2a5-5ca6d48c0f9f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 24 14:16:33.646905 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:16:33.646905 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:16:33.646905 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:16:33.646905 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:16:33.646905 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:16:33.646905 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:16:33.646905 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:16:33.646905 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:16:33.646905 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:16:33.646905 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:16:33.646905 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:16:33.646905 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:16:33.646905 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:16:33.646905 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:16:33.646905 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:16:33.647919 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:16:33.647919 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:16:33.647919 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:16:33.647919 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:16:33.647919 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:16:33.647919 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:16:33.647919 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:16:33.647919 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:16:33.647919 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:16:33.647919 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:16:33.647919 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:16:33.647919 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:16:33.647919 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:16:33.647919 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:16:33.647919 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:16:33.647919 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:16:33.647919 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:16:33.647919 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:16:33.649182 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:16:33.649182 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:16:33.649182 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:16:33.649182 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:16:33.649182 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:16:33.649182 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:16:33.649182 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:16:33.649182 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:16:33.649182 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 24 14:16:33.649182 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:16:33.649182 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 624/1253] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:16:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:16:33.654197 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05a3c4-bbca-4ae2-bb97-9aa6ad75bd77 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:33.714627 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:33.761827 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05a3c4-bbca-4ae2-bb97-9aa6ad75bd77 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:33.797586 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d8092-c874-4869-a2a0-fc49d71528f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:33.798207 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d8092-c874-4869-a2a0-fc49d71528f5 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:33.798484 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d8092-c874-4869-a2a0-fc49d71528f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:33.798755 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d8092-c874-4869-a2a0-fc49d71528f5 None admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:33.799522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d8092-c874-4869-a2a0-fc49d71528f5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:16:33.800352 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d8092-c874-4869-a2a0-fc49d71528f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:33.801795 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d8092-c874-4869-a2a0-fc49d71528f5 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:33.822766 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 630/1254] 10.222.0.52 () {56 vars in 1143 bytes} [Sat Sep 24 14:16:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:16:33.828584 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a6c8c9-08a8-43db-b2a5-5ca6d48c0f9f None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:33.939479 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a6c8c9-08a8-43db-b2a5-5ca6d48c0f9f None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:33.974434 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de352848-fa11-42f1-a14c-54024683288e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:33.975084 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de352848-fa11-42f1-a14c-54024683288e None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:33.975360 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de352848-fa11-42f1-a14c-54024683288e None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:33.975627 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de352848-fa11-42f1-a14c-54024683288e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:34.031683 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:34.045142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de352848-fa11-42f1-a14c-54024683288e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:34.046741 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de352848-fa11-42f1-a14c-54024683288e None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:34.150312 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 625/1255] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:16:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 14:16:34.156159 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d8092-c874-4869-a2a0-fc49d71528f5 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:34.257098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d8092-c874-4869-a2a0-fc49d71528f5 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:34.271145 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:34.291968 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ea21a-6201-4885-9251-1a24eee3692e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:34.292592 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ea21a-6201-4885-9251-1a24eee3692e None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:34.292865 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ea21a-6201-4885-9251-1a24eee3692e None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:34.293138 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ea21a-6201-4885-9251-1a24eee3692e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:34.303306 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ea21a-6201-4885-9251-1a24eee3692e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:34.304741 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ea21a-6201-4885-9251-1a24eee3692e None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:34.319505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-a95ea21a-6201-4885-9251-1a24eee3692e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:16:34.319505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:16:34.319505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:16:34.319505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:16:34.319505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:16:34.319505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:16:34.319505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:16:34.319505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:16:34.319505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:16:34.319505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:16:34.319505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:16:34.319505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:16:34.319505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:16:34.319505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:16:34.319505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:16:34.319505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:16:34.320638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:16:34.320638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:16:34.320638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:16:34.320638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:16:34.320638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:16:34.320638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:16:34.320638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:16:34.320638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:16:34.320638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:16:34.320638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:16:34.320638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:16:34.320638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:16:34.320638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:16:34.320638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:16:34.320638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:16:34.320638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:16:34.320638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:16:34.321725 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:16:34.321725 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:16:34.321725 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:16:34.321725 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 631/1256] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:16:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:16:34.326283 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de352848-fa11-42f1-a14c-54024683288e None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:34.410331 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:34.422204 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de352848-fa11-42f1-a14c-54024683288e None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:34.463722 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7839390-236f-4aea-8908-5310e0430fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:34.464367 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7839390-236f-4aea-8908-5310e0430fa9 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:34.464592 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7839390-236f-4aea-8908-5310e0430fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:34.464868 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7839390-236f-4aea-8908-5310e0430fa9 None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:34.465508 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7839390-236f-4aea-8908-5310e0430fa9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:16:34.465825 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7839390-236f-4aea-8908-5310e0430fa9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:34.467211 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7839390-236f-4aea-8908-5310e0430fa9 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:34.479184 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 626/1257] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:16:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:16:34.484831 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ea21a-6201-4885-9251-1a24eee3692e None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:34.589966 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ea21a-6201-4885-9251-1a24eee3692e None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:34.622473 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e541d64a-4d40-410f-85ab-5216e1c70e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:34.623124 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e541d64a-4d40-410f-85ab-5216e1c70e40 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:34.623403 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e541d64a-4d40-410f-85ab-5216e1c70e40 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:34.623675 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e541d64a-4d40-410f-85ab-5216e1c70e40 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:34.630159 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e541d64a-4d40-410f-85ab-5216e1c70e40 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:34.631624 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e541d64a-4d40-410f-85ab-5216e1c70e40 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:34.643351 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-e541d64a-4d40-410f-85ab-5216e1c70e40 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:16:34.643351 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:16:34.643351 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:16:34.643351 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:16:34.643351 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:16:34.643351 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:16:34.643351 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:16:34.643351 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:16:34.643351 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:16:34.643351 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:16:34.643351 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:16:34.643351 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:16:34.643351 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:16:34.643351 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:16:34.643351 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:16:34.644862 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:16:34.644862 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:16:34.644862 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:16:34.644862 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:16:34.644862 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:16:34.644862 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:16:34.644862 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:16:34.644862 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:16:34.644862 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:16:34.644862 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:16:34.644862 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:16:34.644862 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:16:34.644862 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:16:34.644862 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:16:34.644862 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:16:34.644862 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:16:34.644862 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:16:34.645775 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:16:34.645775 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:16:34.645775 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:16:34.645775 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:16:34.645775 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:16:34.645775 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:16:34.645775 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:16:34.645775 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:16:34.645775 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 632/1258] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:16:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:16:34.650783 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7839390-236f-4aea-8908-5310e0430fa9 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:34.746923 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7839390-236f-4aea-8908-5310e0430fa9 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:34.779332 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3105c1-226b-4947-95b1-ff0d4a80687f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:34.779930 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3105c1-226b-4947-95b1-ff0d4a80687f None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:34.780209 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3105c1-226b-4947-95b1-ff0d4a80687f None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:34.780510 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3105c1-226b-4947-95b1-ff0d4a80687f None admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:34.781194 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3105c1-226b-4947-95b1-ff0d4a80687f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:16:34.781497 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3105c1-226b-4947-95b1-ff0d4a80687f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:34.782942 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3105c1-226b-4947-95b1-ff0d4a80687f None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:34.794952 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 627/1259] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 14:16:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:16:34.800928 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e541d64a-4d40-410f-85ab-5216e1c70e40 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:34.826597 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:34.899649 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e541d64a-4d40-410f-85ab-5216e1c70e40 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:34.931250 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e83669-1042-4756-934e-13ceeb8870b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:34.931838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e83669-1042-4756-934e-13ceeb8870b0 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:34.932112 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e83669-1042-4756-934e-13ceeb8870b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:34.932374 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e83669-1042-4756-934e-13ceeb8870b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:34.933082 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e83669-1042-4756-934e-13ceeb8870b0 None admin] RBAC: Adding query filter params (role.id=d00b78c32dbf4cc7925a55bb621d8fcf, scope.project.id=36a3ae7bab25471fb989136150b6438c, user.id=18fe384ecd454fb298dcc788f41dd44b) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:16:34.933380 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e83669-1042-4756-934e-13ceeb8870b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:34.934785 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e83669-1042-4756-934e-13ceeb8870b0 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:34.939912 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 633/1260] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:16:34 2022] GET /identity/v3/role_assignments?user.id=18fe384ecd454fb298dcc788f41dd44b&scope.project.id=36a3ae7bab25471fb989136150b6438c&role.id=d00b78c32dbf4cc7925a55bb621d8fcf => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:16:37.076025 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf0991-b496-4c2a-9449-5a8160ab7e84 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:37.076544 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf0991-b496-4c2a-9449-5a8160ab7e84 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:37.076628 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf0991-b496-4c2a-9449-5a8160ab7e84 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:37.077913 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 628/1261] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:16:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:16:37.083711 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6f401-2fd8-462d-b7d4-5c1154ab8790 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:37.084130 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6f401-2fd8-462d-b7d4-5c1154ab8790 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:37.084222 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6f401-2fd8-462d-b7d4-5c1154ab8790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:37.102156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-6df6f401-2fd8-462d-b7d4-5c1154ab8790 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:16:37.103348 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-6df6f401-2fd8-462d-b7d4-5c1154ab8790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:16:37.167236 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 634/1262] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:16:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:16:37.432541 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf0991-b496-4c2a-9449-5a8160ab7e84 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:37.443343 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:37.531264 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf0991-b496-4c2a-9449-5a8160ab7e84 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:37.574967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d1ae2-71bf-4fa8-8fe4-c125bab36615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:37.575673 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d1ae2-71bf-4fa8-8fe4-c125bab36615 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:37.575996 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d1ae2-71bf-4fa8-8fe4-c125bab36615 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:37.576399 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d1ae2-71bf-4fa8-8fe4-c125bab36615 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:37.589421 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900d1ae2-71bf-4fa8-8fe4-c125bab36615 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:16:37.595671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-900d1ae2-71bf-4fa8-8fe4-c125bab36615 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:16:37.595671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:16:37.595671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:16:37.595671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:16:37.595671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:16:37.595671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:16:37.595671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:16:37.595671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:16:37.595671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:16:37.595671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:16:37.595671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:16:37.595671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:16:37.595671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:16:37.595671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:16:37.595671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:16:37.595671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:16:37.597444 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:16:37.597444 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:16:37.597444 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:16:37.597444 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:16:37.597444 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:16:37.597444 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:16:37.597444 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:16:37.597444 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:16:37.597444 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:16:37.597444 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:16:37.597444 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:16:37.597444 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:16:37.597444 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:16:37.597444 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:16:37.597444 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:16:37.597444 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:16:37.598533 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:16:37.598533 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:16:37.598533 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:16:37.598533 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:16:37.598533 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:16:37.598533 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:16:37.598533 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:16:37.598533 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:16:37.598533 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:16:37.598533 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:16:37.598533 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:16:37.598533 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:16:37.598533 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 629/1263] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 14:16:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:16:37.604465 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df6f401-2fd8-462d-b7d4-5c1154ab8790 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:37.622774 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:37.701326 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df6f401-2fd8-462d-b7d4-5c1154ab8790 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:37.735492 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524aaffb-273d-4711-b320-9971bc8443ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:37.736256 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524aaffb-273d-4711-b320-9971bc8443ba None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:37.736566 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524aaffb-273d-4711-b320-9971bc8443ba None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:37.736836 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524aaffb-273d-4711-b320-9971bc8443ba None admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:37.737500 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524aaffb-273d-4711-b320-9971bc8443ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:16:37.737802 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524aaffb-273d-4711-b320-9971bc8443ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:37.739199 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524aaffb-273d-4711-b320-9971bc8443ba None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:37.747127 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 635/1264] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:16:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:16:37.753512 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d1ae2-71bf-4fa8-8fe4-c125bab36615 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:37.819686 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:37.855265 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d1ae2-71bf-4fa8-8fe4-c125bab36615 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:37.888201 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc80f7ad-4418-45f2-be0a-64d0bcf1980e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:37.888816 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc80f7ad-4418-45f2-be0a-64d0bcf1980e None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:37.889112 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc80f7ad-4418-45f2-be0a-64d0bcf1980e None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:37.889380 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc80f7ad-4418-45f2-be0a-64d0bcf1980e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:37.899736 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc80f7ad-4418-45f2-be0a-64d0bcf1980e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:37.901167 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc80f7ad-4418-45f2-be0a-64d0bcf1980e None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:37.915480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-dc80f7ad-4418-45f2-be0a-64d0bcf1980e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:16:37.915480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:16:37.915480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:16:37.915480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:16:37.915480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:16:37.915480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:16:37.915480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:16:37.915480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:16:37.915480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:16:37.915480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:16:37.915480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:16:37.915480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:16:37.915480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:16:37.915480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:16:37.915480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:16:37.915480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:16:37.916538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:16:37.916538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:16:37.916538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:16:37.916538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:16:37.916538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:16:37.916538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:16:37.916538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:16:37.916538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:16:37.916538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:16:37.916538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:16:37.916538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:16:37.916538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:16:37.916538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:16:37.916538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:16:37.916538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:16:37.916538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:16:37.916538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:16:37.917677 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:16:37.917677 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:16:37.917677 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:16:37.917677 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 630/1265] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:16:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:16:37.922311 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524aaffb-273d-4711-b320-9971bc8443ba None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:38.000033 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:38.019751 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524aaffb-273d-4711-b320-9971bc8443ba None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:38.052238 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce8abf-1ca8-4773-8fc4-1490d06bdf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:38.052827 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce8abf-1ca8-4773-8fc4-1490d06bdf45 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:38.053101 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce8abf-1ca8-4773-8fc4-1490d06bdf45 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:38.053377 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce8abf-1ca8-4773-8fc4-1490d06bdf45 None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:38.054010 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ce8abf-1ca8-4773-8fc4-1490d06bdf45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:16:38.054314 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ce8abf-1ca8-4773-8fc4-1490d06bdf45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:38.055703 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ce8abf-1ca8-4773-8fc4-1490d06bdf45 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:38.067403 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 636/1266] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:16:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:16:38.073012 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc80f7ad-4418-45f2-be0a-64d0bcf1980e None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:38.168359 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc80f7ad-4418-45f2-be0a-64d0bcf1980e None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:38.187382 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:38.201813 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dce989-84f7-4a14-8b93-72c717d6aab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:38.202421 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dce989-84f7-4a14-8b93-72c717d6aab7 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:38.202700 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dce989-84f7-4a14-8b93-72c717d6aab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:38.203000 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dce989-84f7-4a14-8b93-72c717d6aab7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:38.210128 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dce989-84f7-4a14-8b93-72c717d6aab7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:38.211581 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dce989-84f7-4a14-8b93-72c717d6aab7 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:38.222322 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-38dce989-84f7-4a14-8b93-72c717d6aab7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 24 14:16:38.222322 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:16:38.222322 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:16:38.222322 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:16:38.222322 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:16:38.222322 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:16:38.222322 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:16:38.222322 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:16:38.222322 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:16:38.222322 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:16:38.222322 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:16:38.222322 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:16:38.222322 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:16:38.222322 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:16:38.222322 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:16:38.222322 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:16:38.228040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:16:38.228040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:16:38.228040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:16:38.228040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:16:38.228040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:16:38.228040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:16:38.228040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:16:38.228040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:16:38.228040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:16:38.228040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:16:38.228040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:16:38.228040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:16:38.228040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:16:38.228040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:16:38.228040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:16:38.228040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:16:38.228040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:16:38.228040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:16:38.229268 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:16:38.229268 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:16:38.229268 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:16:38.229268 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:16:38.229268 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:16:38.229268 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:16:38.229268 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:16:38.229268 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:16:38.229268 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 24 14:16:38.229268 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:16:38.229268 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 631/1267] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:16:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:16:38.229982 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce8abf-1ca8-4773-8fc4-1490d06bdf45 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:38.329358 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce8abf-1ca8-4773-8fc4-1490d06bdf45 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:38.361585 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9a4243-2689-4ff1-a06c-590d12253f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:38.362673 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a4243-2689-4ff1-a06c-590d12253f67 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:38.362981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a4243-2689-4ff1-a06c-590d12253f67 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:38.363254 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a4243-2689-4ff1-a06c-590d12253f67 None admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:38.363970 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9a4243-2689-4ff1-a06c-590d12253f67 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:16:38.364271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9a4243-2689-4ff1-a06c-590d12253f67 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:38.365695 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9a4243-2689-4ff1-a06c-590d12253f67 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:38.373985 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:38.383732 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 637/1268] 10.222.0.52 () {56 vars in 1143 bytes} [Sat Sep 24 14:16:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:16:38.389541 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dce989-84f7-4a14-8b93-72c717d6aab7 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:38.485034 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dce989-84f7-4a14-8b93-72c717d6aab7 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:38.516784 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fc345e-d3fb-41bf-8c13-06fa124bbc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:38.517386 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc345e-d3fb-41bf-8c13-06fa124bbc91 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:38.517661 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc345e-d3fb-41bf-8c13-06fa124bbc91 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:38.517930 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc345e-d3fb-41bf-8c13-06fa124bbc91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:38.527942 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fc345e-d3fb-41bf-8c13-06fa124bbc91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:38.529551 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fc345e-d3fb-41bf-8c13-06fa124bbc91 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:38.547713 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-57fc345e-d3fb-41bf-8c13-06fa124bbc91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:16:38.547713 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:16:38.547713 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:16:38.547713 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:16:38.547713 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:16:38.547713 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:16:38.547713 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:16:38.547713 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:16:38.547713 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:16:38.547713 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:16:38.547713 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:16:38.547713 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:16:38.547713 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:16:38.547713 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:16:38.547713 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:16:38.547713 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:16:38.549384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:16:38.549384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:16:38.549384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:16:38.549384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:16:38.549384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:16:38.549384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:16:38.549384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:16:38.549384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:16:38.549384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:16:38.549384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:16:38.549384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:16:38.549384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:16:38.549384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:16:38.549384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:16:38.549384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:16:38.549384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:16:38.549384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:16:38.550318 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:16:38.550318 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:16:38.550318 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:16:38.550318 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 632/1269] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:16:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:16:38.556056 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9a4243-2689-4ff1-a06c-590d12253f67 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:38.652079 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9a4243-2689-4ff1-a06c-590d12253f67 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:38.686023 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad40d00-8ab6-489b-9a8c-590b676f6473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:38.686616 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad40d00-8ab6-489b-9a8c-590b676f6473 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:38.686921 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad40d00-8ab6-489b-9a8c-590b676f6473 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:38.687197 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad40d00-8ab6-489b-9a8c-590b676f6473 None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:38.687833 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad40d00-8ab6-489b-9a8c-590b676f6473 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:16:38.688135 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad40d00-8ab6-489b-9a8c-590b676f6473 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:38.689509 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad40d00-8ab6-489b-9a8c-590b676f6473 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:38.700388 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 638/1270] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:16:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:16:38.706170 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fc345e-d3fb-41bf-8c13-06fa124bbc91 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:38.736951 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:38.804330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fc345e-d3fb-41bf-8c13-06fa124bbc91 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:38.836695 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd3d48-7dd7-4a45-90ce-4ee5424a0d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:38.837326 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd3d48-7dd7-4a45-90ce-4ee5424a0d24 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:38.837600 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd3d48-7dd7-4a45-90ce-4ee5424a0d24 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:38.837865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd3d48-7dd7-4a45-90ce-4ee5424a0d24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:38.845140 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefd3d48-7dd7-4a45-90ce-4ee5424a0d24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:38.847434 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefd3d48-7dd7-4a45-90ce-4ee5424a0d24 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:38.872384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-eefd3d48-7dd7-4a45-90ce-4ee5424a0d24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:16:38.872384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:16:38.872384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:16:38.872384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:16:38.872384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:16:38.872384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:16:38.872384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:16:38.872384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:16:38.872384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:16:38.872384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:16:38.872384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:16:38.872384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:16:38.872384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:16:38.872384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:16:38.872384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:16:38.873480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:16:38.873480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:16:38.873480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:16:38.873480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:16:38.873480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:16:38.873480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:16:38.873480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:16:38.873480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:16:38.873480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:16:38.873480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:16:38.873480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:16:38.873480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:16:38.873480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:16:38.873480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:16:38.873480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:16:38.873480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:16:38.873480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:16:38.874761 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:16:38.874761 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:16:38.874761 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:16:38.874761 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:16:38.874761 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:16:38.874761 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:16:38.874761 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:16:38.874761 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:16:38.874761 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 633/1271] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:16:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 14:16:38.880000 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad40d00-8ab6-489b-9a8c-590b676f6473 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:38.936128 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:38.976530 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad40d00-8ab6-489b-9a8c-590b676f6473 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:39.009541 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869bfdac-ef6f-4b6f-a590-8353545879f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:39.010143 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869bfdac-ef6f-4b6f-a590-8353545879f3 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:39.010424 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869bfdac-ef6f-4b6f-a590-8353545879f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:39.010695 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869bfdac-ef6f-4b6f-a590-8353545879f3 None admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:39.011405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869bfdac-ef6f-4b6f-a590-8353545879f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:16:39.011708 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869bfdac-ef6f-4b6f-a590-8353545879f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:39.013131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869bfdac-ef6f-4b6f-a590-8353545879f3 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:39.024679 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 639/1272] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 14:16:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:16:39.030579 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd3d48-7dd7-4a45-90ce-4ee5424a0d24 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:39.117695 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:39.126242 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd3d48-7dd7-4a45-90ce-4ee5424a0d24 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:39.157919 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5740e978-8577-46bc-a09c-2c47e0957790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:39.159337 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740e978-8577-46bc-a09c-2c47e0957790 None admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:39.159641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740e978-8577-46bc-a09c-2c47e0957790 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:39.159906 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740e978-8577-46bc-a09c-2c47e0957790 None admin] PATH_INFO: `/v3/projects/36a3ae7bab25471fb989136150b6438c/users/18fe384ecd454fb298dcc788f41dd44b/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:39.161730 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5740e978-8577-46bc-a09c-2c47e0957790 None admin] RBAC: Authorizing `identity:create_grant(project_id=36a3ae7bab25471fb989136150b6438c, user_id=18fe384ecd454fb298dcc788f41dd44b, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:39.163159 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5740e978-8577-46bc-a09c-2c47e0957790 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:39.175487 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 634/1273] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 14:16:39 2022] PUT /identity/v3/projects/36a3ae7bab25471fb989136150b6438c/users/18fe384ecd454fb298dcc788f41dd44b/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:16:41.301972 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b9797e-14ac-4bf3-822d-851fc4d89d7c None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:41.302527 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b9797e-14ac-4bf3-822d-851fc4d89d7c None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:41.302527 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b9797e-14ac-4bf3-822d-851fc4d89d7c None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:41.303888 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 640/1274] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:16:41.309015 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8400c-8edf-4b34-92a6-c9d8e54c4fbb None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:41.309423 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8400c-8edf-4b34-92a6-c9d8e54c4fbb None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:41.309561 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8400c-8edf-4b34-92a6-c9d8e54c4fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:41.327741 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-17f8400c-8edf-4b34-92a6-c9d8e54c4fbb None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:16:41.328899 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-17f8400c-8edf-4b34-92a6-c9d8e54c4fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:16:41.391485 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 635/1275] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:16:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:16:41.668635 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b9797e-14ac-4bf3-822d-851fc4d89d7c None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:41.767767 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b9797e-14ac-4bf3-822d-851fc4d89d7c None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:41.785087 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:41.802041 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ee73c-020c-448f-a908-2e3b8a2352be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:41.802657 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ee73c-020c-448f-a908-2e3b8a2352be None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:41.802970 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ee73c-020c-448f-a908-2e3b8a2352be None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:41.803246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ee73c-020c-448f-a908-2e3b8a2352be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:41.813472 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05ee73c-020c-448f-a908-2e3b8a2352be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 14:16:41.818886 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-c05ee73c-020c-448f-a908-2e3b8a2352be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:16:41.818886 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:16:41.818886 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:16:41.818886 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:16:41.818886 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:16:41.818886 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:16:41.818886 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:16:41.818886 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:16:41.818886 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:16:41.818886 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:16:41.818886 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:16:41.818886 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:16:41.818886 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 14:16:41.818886 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 14:16:41.818886 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 14:16:41.818886 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 14:16:41.819981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 14:16:41.819981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 14:16:41.819981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 14:16:41.819981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 14:16:41.819981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:16:41.819981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:16:41.819981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:16:41.819981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:16:41.819981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:16:41.819981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:16:41.819981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:16:41.819981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:16:41.819981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:16:41.819981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:16:41.819981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:16:41.819981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:16:41.820819 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:16:41.820819 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:16:41.820819 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:16:41.820819 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:16:41.820819 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 14:16:41.820819 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 14:16:41.820819 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 14:16:41.820819 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 14:16:41.820819 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 14:16:41.820819 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 14:16:41.820819 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 14:16:41.820819 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:16:41.820819 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 641/1276] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 14:16:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:16:41.825962 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f8400c-8edf-4b34-92a6-c9d8e54c4fbb None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:41.904282 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:41.922533 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f8400c-8edf-4b34-92a6-c9d8e54c4fbb None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:41.955161 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9200de-7aa9-4b2c-9d5b-9f58b9173674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:41.955780 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9200de-7aa9-4b2c-9d5b-9f58b9173674 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:41.956054 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9200de-7aa9-4b2c-9d5b-9f58b9173674 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:41.956324 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9200de-7aa9-4b2c-9d5b-9f58b9173674 None admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:41.956975 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9200de-7aa9-4b2c-9d5b-9f58b9173674 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:16:41.957274 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9200de-7aa9-4b2c-9d5b-9f58b9173674 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:41.958619 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9200de-7aa9-4b2c-9d5b-9f58b9173674 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:41.966185 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 636/1277] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 14:16:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:16:41.972061 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ee73c-020c-448f-a908-2e3b8a2352be None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:42.069386 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ee73c-020c-448f-a908-2e3b8a2352be None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:42.102215 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66638626-fb19-42db-b0d1-1714153c8513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:42.102844 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66638626-fb19-42db-b0d1-1714153c8513 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:42.103136 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66638626-fb19-42db-b0d1-1714153c8513 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:42.103406 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66638626-fb19-42db-b0d1-1714153c8513 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:42.113611 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66638626-fb19-42db-b0d1-1714153c8513 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:42.115071 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66638626-fb19-42db-b0d1-1714153c8513 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:42.129699 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-66638626-fb19-42db-b0d1-1714153c8513 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:16:42.129699 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:16:42.129699 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:16:42.129699 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:16:42.129699 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:16:42.129699 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:16:42.129699 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:16:42.129699 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:16:42.129699 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:16:42.129699 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:16:42.129699 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:16:42.129699 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:16:42.129699 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:16:42.129699 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:16:42.129699 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:16:42.129699 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:16:42.130790 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:16:42.130790 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:16:42.130790 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:16:42.130790 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:16:42.130790 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:16:42.130790 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:16:42.130790 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:16:42.130790 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:16:42.130790 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:16:42.130790 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:16:42.130790 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:16:42.130790 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:16:42.130790 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:16:42.130790 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:16:42.130790 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:16:42.130790 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:16:42.130790 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:16:42.131879 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:16:42.131879 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:16:42.131879 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:16:42.131879 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 642/1278] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:16:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:16:42.136371 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9200de-7aa9-4b2c-9d5b-9f58b9173674 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:42.231381 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9200de-7aa9-4b2c-9d5b-9f58b9173674 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:42.263281 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10916c-4f7e-43dc-83a4-c7f08af56d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:42.263863 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10916c-4f7e-43dc-83a4-c7f08af56d7d None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:42.264142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10916c-4f7e-43dc-83a4-c7f08af56d7d None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:42.264408 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10916c-4f7e-43dc-83a4-c7f08af56d7d None admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:42.265028 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f10916c-4f7e-43dc-83a4-c7f08af56d7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:16:42.265320 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f10916c-4f7e-43dc-83a4-c7f08af56d7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:42.266656 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f10916c-4f7e-43dc-83a4-c7f08af56d7d None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:42.278187 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 637/1279] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:16:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:16:42.284420 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66638626-fb19-42db-b0d1-1714153c8513 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:42.315219 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:42.382205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66638626-fb19-42db-b0d1-1714153c8513 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:42.414703 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8e404c-c7c6-4e91-b5ac-efc3f36a3452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:42.415334 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e404c-c7c6-4e91-b5ac-efc3f36a3452 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:42.415607 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e404c-c7c6-4e91-b5ac-efc3f36a3452 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:42.415873 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e404c-c7c6-4e91-b5ac-efc3f36a3452 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:42.422978 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8e404c-c7c6-4e91-b5ac-efc3f36a3452 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:42.424420 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8e404c-c7c6-4e91-b5ac-efc3f36a3452 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:42.435151 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-be8e404c-c7c6-4e91-b5ac-efc3f36a3452 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 24 14:16:42.435151 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:16:42.435151 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:16:42.435151 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:16:42.435151 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:16:42.435151 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:16:42.435151 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:16:42.435151 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:16:42.435151 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:16:42.435151 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:16:42.435151 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:16:42.435151 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:16:42.435151 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 14:16:42.435151 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 14:16:42.435151 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 14:16:42.435151 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 14:16:42.436295 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:16:42.436295 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:16:42.436295 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 14:16:42.436295 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:16:42.436295 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 14:16:42.436295 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 14:16:42.436295 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:16:42.436295 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:16:42.436295 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:16:42.436295 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:16:42.436295 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:16:42.436295 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:16:42.436295 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:16:42.436295 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:16:42.436295 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:16:42.436295 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:16:42.436295 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:16:42.436295 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:16:42.437552 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:16:42.437552 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:16:42.437552 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 14:16:42.437552 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 14:16:42.437552 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 14:16:42.437552 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 14:16:42.437552 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 14:16:42.437552 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 14:16:42.437552 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 24 14:16:42.437552 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:16:42.437552 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 643/1280] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:16:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:16:42.442718 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10916c-4f7e-43dc-83a4-c7f08af56d7d None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:42.446503 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:42.538857 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10916c-4f7e-43dc-83a4-c7f08af56d7d None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:42.570721 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d1ec5-77c9-4795-9120-0a00c2b14e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:42.571351 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d1ec5-77c9-4795-9120-0a00c2b14e9e None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:42.571629 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d1ec5-77c9-4795-9120-0a00c2b14e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:42.571898 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d1ec5-77c9-4795-9120-0a00c2b14e9e None admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:42.572627 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d1ec5-77c9-4795-9120-0a00c2b14e9e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:16:42.572927 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d1ec5-77c9-4795-9120-0a00c2b14e9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:42.574347 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d1ec5-77c9-4795-9120-0a00c2b14e9e None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:42.591206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 638/1281] 10.222.0.52 () {56 vars in 1143 bytes} [Sat Sep 24 14:16:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:16:42.597542 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8e404c-c7c6-4e91-b5ac-efc3f36a3452 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:42.683834 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:42.693407 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8e404c-c7c6-4e91-b5ac-efc3f36a3452 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:42.726390 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2523f2d-5b51-4d8c-b57c-7bb472795143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:42.727009 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2523f2d-5b51-4d8c-b57c-7bb472795143 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:42.727287 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2523f2d-5b51-4d8c-b57c-7bb472795143 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:42.727557 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2523f2d-5b51-4d8c-b57c-7bb472795143 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:42.791393 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2523f2d-5b51-4d8c-b57c-7bb472795143 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:42.792770 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2523f2d-5b51-4d8c-b57c-7bb472795143 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:42.882333 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:42.892729 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 644/1282] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 14:16:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 14:16:42.899078 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d1ec5-77c9-4795-9120-0a00c2b14e9e None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:42.958738 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:42.994819 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d1ec5-77c9-4795-9120-0a00c2b14e9e None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:43.027549 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcbf37d-ecfc-4a2d-af53-09e62e65f9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:43.028164 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbf37d-ecfc-4a2d-af53-09e62e65f9d2 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:43.028441 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbf37d-ecfc-4a2d-af53-09e62e65f9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:43.028711 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbf37d-ecfc-4a2d-af53-09e62e65f9d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:43.038614 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcbf37d-ecfc-4a2d-af53-09e62e65f9d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:43.040068 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcbf37d-ecfc-4a2d-af53-09e62e65f9d2 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:43.054020 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-cfcbf37d-ecfc-4a2d-af53-09e62e65f9d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:16:43.054020 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:16:43.054020 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:16:43.054020 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:16:43.054020 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:16:43.054020 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:16:43.054020 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:16:43.054020 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:16:43.054020 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:16:43.054020 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:16:43.054020 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:16:43.054020 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:16:43.054020 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 14:16:43.054020 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 14:16:43.054020 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 14:16:43.054020 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 14:16:43.055142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:16:43.055142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:16:43.055142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:16:43.055142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:16:43.055142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:16:43.055142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:16:43.055142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:16:43.055142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:16:43.055142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:16:43.055142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:16:43.055142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:16:43.055142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:16:43.055142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:16:43.055142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:16:43.055142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:16:43.055142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:16:43.055142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 14:16:43.056199 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 14:16:43.056199 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 14:16:43.056199 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:16:43.056199 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 639/1283] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:16:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:16:43.060716 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2523f2d-5b51-4d8c-b57c-7bb472795143 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:43.157496 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2523f2d-5b51-4d8c-b57c-7bb472795143 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:43.191153 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b291c4-dea8-47f0-9b3c-acb3a2b75731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:43.191749 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b291c4-dea8-47f0-9b3c-acb3a2b75731 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:43.192025 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b291c4-dea8-47f0-9b3c-acb3a2b75731 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:43.192293 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b291c4-dea8-47f0-9b3c-acb3a2b75731 None admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:43.192923 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b291c4-dea8-47f0-9b3c-acb3a2b75731 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:16:43.193222 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b291c4-dea8-47f0-9b3c-acb3a2b75731 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:43.194589 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b291c4-dea8-47f0-9b3c-acb3a2b75731 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:43.205371 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 645/1284] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 14:16:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:16:43.211589 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcbf37d-ecfc-4a2d-af53-09e62e65f9d2 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:43.307298 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcbf37d-ecfc-4a2d-af53-09e62e65f9d2 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:43.320580 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:43.340366 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012591c0-4eb0-44e9-848b-b4d2a84993a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:43.340972 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012591c0-4eb0-44e9-848b-b4d2a84993a3 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:43.341253 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012591c0-4eb0-44e9-848b-b4d2a84993a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:43.341519 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012591c0-4eb0-44e9-848b-b4d2a84993a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:43.348400 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012591c0-4eb0-44e9-848b-b4d2a84993a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:43.349802 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012591c0-4eb0-44e9-848b-b4d2a84993a3 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:43.360512 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application [None req-012591c0-4eb0-44e9-848b-b4d2a84993a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:16:43.360512 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:16:43.360512 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:16:43.360512 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:16:43.360512 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:16:43.360512 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:16:43.360512 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:16:43.360512 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:16:43.360512 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:16:43.360512 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:16:43.360512 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:16:43.360512 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:16:43.360512 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 14:16:43.360512 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 14:16:43.360512 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 14:16:43.361536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 14:16:43.361536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:16:43.361536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:16:43.361536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:16:43.361536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:16:43.361536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:16:43.361536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:16:43.361536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:16:43.361536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application with Lock( Sep 24 14:16:43.361536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:16:43.361536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:16:43.361536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:16:43.361536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:16:43.361536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:16:43.361536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:16:43.361536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:16:43.361536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:16:43.362689 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 14:16:43.362689 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 14:16:43.362689 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 14:16:43.362689 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 14:16:43.362689 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 14:16:43.362689 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 14:16:43.362689 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 14:16:43.362689 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: ERROR keystone.server.flask.application Sep 24 14:16:43.362689 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 640/1285] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 14:16:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:16:43.367305 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b291c4-dea8-47f0-9b3c-acb3a2b75731 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:43.416479 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:43.464263 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b291c4-dea8-47f0-9b3c-acb3a2b75731 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:43.496591 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad0568-012e-4151-bccb-950f20d0af65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:43.497174 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad0568-012e-4151-bccb-950f20d0af65 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:43.497447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad0568-012e-4151-bccb-950f20d0af65 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:43.497714 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad0568-012e-4151-bccb-950f20d0af65 None admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:43.498385 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ad0568-012e-4151-bccb-950f20d0af65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:16:43.498685 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ad0568-012e-4151-bccb-950f20d0af65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:43.500160 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ad0568-012e-4151-bccb-950f20d0af65 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:43.512055 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 646/1286] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 14:16:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:16:43.518549 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012591c0-4eb0-44e9-848b-b4d2a84993a3 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:43.615138 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012591c0-4eb0-44e9-848b-b4d2a84993a3 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:43.646822 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bae9054-ec54-4bb9-93b4-a7095883f043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:43.647428 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae9054-ec54-4bb9-93b4-a7095883f043 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:43.647698 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae9054-ec54-4bb9-93b4-a7095883f043 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:43.647962 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae9054-ec54-4bb9-93b4-a7095883f043 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:43.648672 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bae9054-ec54-4bb9-93b4-a7095883f043 None admin] RBAC: Adding query filter params (role.id=d00b78c32dbf4cc7925a55bb621d8fcf, scope.project.id=36a3ae7bab25471fb989136150b6438c, user.id=18fe384ecd454fb298dcc788f41dd44b) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:16:43.648970 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bae9054-ec54-4bb9-93b4-a7095883f043 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:43.650375 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bae9054-ec54-4bb9-93b4-a7095883f043 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:43.655849 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 641/1287] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 14:16:43 2022] GET /identity/v3/role_assignments?user.id=18fe384ecd454fb298dcc788f41dd44b&scope.project.id=36a3ae7bab25471fb989136150b6438c&role.id=d00b78c32dbf4cc7925a55bb621d8fcf => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:16:45.775420 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51535a7e-e830-46f9-a32f-fa3d6c6428de None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:45.775983 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51535a7e-e830-46f9-a32f-fa3d6c6428de None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:45.775983 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51535a7e-e830-46f9-a32f-fa3d6c6428de None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:45.777302 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 647/1288] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:16:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:16:45.782385 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce938f-2a56-45fc-b5aa-184c7f848ac9 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:45.782769 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce938f-2a56-45fc-b5aa-184c7f848ac9 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:45.782968 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce938f-2a56-45fc-b5aa-184c7f848ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:45.801698 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-11ce938f-2a56-45fc-b5aa-184c7f848ac9 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:16:45.802893 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-11ce938f-2a56-45fc-b5aa-184c7f848ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:16:45.831632 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:45.868945 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 642/1289] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:16:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:16:46.135372 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51535a7e-e830-46f9-a32f-fa3d6c6428de None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:46.233165 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51535a7e-e830-46f9-a32f-fa3d6c6428de None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:46.243987 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:46.266720 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a20c74-f0ef-4be9-b614-68d5a1ea5a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:46.267359 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a20c74-f0ef-4be9-b614-68d5a1ea5a59 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:46.267636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a20c74-f0ef-4be9-b614-68d5a1ea5a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:46.268407 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a20c74-f0ef-4be9-b614-68d5a1ea5a59 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:46.268991 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a20c74-f0ef-4be9-b614-68d5a1ea5a59 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:46.270349 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a20c74-f0ef-4be9-b614-68d5a1ea5a59 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:46.278863 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-a3a20c74-f0ef-4be9-b614-68d5a1ea5a59 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 14:16:46.278863 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:16:46.278863 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:16:46.278863 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:16:46.278863 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:16:46.278863 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:16:46.278863 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:16:46.278863 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:16:46.278863 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 14:16:46.278863 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:16:46.278863 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:16:46.278863 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:16:46.278863 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:16:46.278863 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:16:46.278863 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:16:46.278863 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:16:46.278863 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:16:46.280173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:16:46.280173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:16:46.280173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:16:46.280173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:16:46.280173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:16:46.280173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:16:46.280173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:16:46.280173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:16:46.280173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:16:46.280173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:16:46.280173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:16:46.280173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:16:46.280173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:16:46.280173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:16:46.280173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:16:46.280173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:16:46.281368 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:16:46.281368 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:16:46.281368 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:16:46.281368 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:16:46.281368 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:16:46.281368 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:16:46.281368 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:16:46.281368 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:16:46.281368 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:16:46.281368 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:16:46.281368 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:16:46.281368 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:16:46.281368 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 14:16:46.281368 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:16:46.281368 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 648/1290] 10.222.0.52 () {56 vars in 1117 bytes} [Sat Sep 24 14:16:46 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:16:46.286681 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ce938f-2a56-45fc-b5aa-184c7f848ac9 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:46.380280 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ce938f-2a56-45fc-b5aa-184c7f848ac9 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:46.413420 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3786b7d-d259-4f41-bc89-8fb8fe0c60f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:46.413953 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3786b7d-d259-4f41-bc89-8fb8fe0c60f3 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:46.414230 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3786b7d-d259-4f41-bc89-8fb8fe0c60f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:46.414500 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3786b7d-d259-4f41-bc89-8fb8fe0c60f3 None admin] PATH_INFO: `/v3/services` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:46.415147 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3786b7d-d259-4f41-bc89-8fb8fe0c60f3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:16:46.415445 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3786b7d-d259-4f41-bc89-8fb8fe0c60f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:46.416800 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3786b7d-d259-4f41-bc89-8fb8fe0c60f3 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:46.421397 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 643/1291] 10.222.0.52 () {56 vars in 1116 bytes} [Sat Sep 24 14:16:46 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:16:46.427014 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a20c74-f0ef-4be9-b614-68d5a1ea5a59 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:46.522754 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a20c74-f0ef-4be9-b614-68d5a1ea5a59 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:46.558789 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240f1f6-b7c3-44c6-abd8-e7e5ff4d16c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:46.559521 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240f1f6-b7c3-44c6-abd8-e7e5ff4d16c1 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:46.559874 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240f1f6-b7c3-44c6-abd8-e7e5ff4d16c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:46.560225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240f1f6-b7c3-44c6-abd8-e7e5ff4d16c1 None admin] PATH_INFO: `/v3/services` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:46.560944 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240f1f6-b7c3-44c6-abd8-e7e5ff4d16c1 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:16:46.561352 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240f1f6-b7c3-44c6-abd8-e7e5ff4d16c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:46.562947 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240f1f6-b7c3-44c6-abd8-e7e5ff4d16c1 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:46.568123 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 649/1292] 10.222.0.52 () {56 vars in 1116 bytes} [Sat Sep 24 14:16:46 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:16:48.693891 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80295ea-38f8-43e2-b914-6c029bc8ea1b None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:48.694418 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80295ea-38f8-43e2-b914-6c029bc8ea1b None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:48.694567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80295ea-38f8-43e2-b914-6c029bc8ea1b None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:48.696120 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 644/1293] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:16:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:16:48.701570 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be24193-4bc1-49f6-942e-413344cf2045 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:48.701981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be24193-4bc1-49f6-942e-413344cf2045 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:48.702105 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be24193-4bc1-49f6-942e-413344cf2045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:48.720143 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-1be24193-4bc1-49f6-942e-413344cf2045 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:16:48.721352 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-1be24193-4bc1-49f6-942e-413344cf2045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:16:48.749795 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:48.785929 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 650/1294] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:16:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:16:49.059506 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80295ea-38f8-43e2-b914-6c029bc8ea1b None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:49.090419 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:49.158952 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80295ea-38f8-43e2-b914-6c029bc8ea1b None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:49.201656 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e7cc0e-cfe4-4b49-b9f5-b9436764c9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:49.202697 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7cc0e-cfe4-4b49-b9f5-b9436764c9f9 None admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:49.203250 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7cc0e-cfe4-4b49-b9f5-b9436764c9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:49.203759 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7cc0e-cfe4-4b49-b9f5-b9436764c9f9 None admin] PATH_INFO: `/v3/services` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:49.204714 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e7cc0e-cfe4-4b49-b9f5-b9436764c9f9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:49.206660 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e7cc0e-cfe4-4b49-b9f5-b9436764c9f9 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:49.222561 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 645/1295] 10.222.0.52 () {60 vars in 1141 bytes} [Sat Sep 24 14:16:49 2022] POST /identity/v3/services => generated 246 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:16:51.363090 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901e686-b543-4c2a-85e0-f0e0e4bc6abf None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:51.363621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901e686-b543-4c2a-85e0-f0e0e4bc6abf None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:51.363621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901e686-b543-4c2a-85e0-f0e0e4bc6abf None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:51.364976 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 651/1296] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:16:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:16:51.370152 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa094bda-b8c4-4483-8ad0-a75e9af97a7b None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:51.370591 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa094bda-b8c4-4483-8ad0-a75e9af97a7b None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:51.370698 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa094bda-b8c4-4483-8ad0-a75e9af97a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:51.388397 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-fa094bda-b8c4-4483-8ad0-a75e9af97a7b None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:16:51.389564 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-fa094bda-b8c4-4483-8ad0-a75e9af97a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:16:51.458858 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 646/1297] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:16:51 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 14:16:51.735734 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b901e686-b543-4c2a-85e0-f0e0e4bc6abf None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:51.836768 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b901e686-b543-4c2a-85e0-f0e0e4bc6abf None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:51.869797 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e34f3ab-04d2-408c-8b0b-a7e619216cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:51.870412 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34f3ab-04d2-408c-8b0b-a7e619216cef None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:51.870686 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34f3ab-04d2-408c-8b0b-a7e619216cef None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:51.870982 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34f3ab-04d2-408c-8b0b-a7e619216cef None admin] PATH_INFO: `/v3/services/placement` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:51.871558 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e34f3ab-04d2-408c-8b0b-a7e619216cef None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:51.872911 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e34f3ab-04d2-408c-8b0b-a7e619216cef None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:51.881415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-5e34f3ab-04d2-408c-8b0b-a7e619216cef None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 14:16:51.881415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:16:51.881415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:16:51.881415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:16:51.881415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:16:51.881415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:16:51.881415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:16:51.881415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:16:51.881415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 14:16:51.881415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:16:51.881415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:16:51.881415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:16:51.881415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:16:51.881415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:16:51.881415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:16:51.881415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:16:51.881415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:16:51.882549 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:16:51.882549 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:16:51.882549 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:16:51.882549 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:16:51.882549 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:16:51.882549 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:16:51.882549 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:16:51.882549 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:16:51.882549 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:16:51.882549 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:16:51.882549 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:16:51.882549 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:16:51.882549 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:16:51.882549 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:16:51.882549 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:16:51.882549 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:16:51.883922 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:16:51.883922 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:16:51.883922 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:16:51.883922 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:16:51.883922 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:16:51.883922 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:16:51.883922 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:16:51.883922 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:16:51.883922 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:16:51.883922 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:16:51.883922 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:16:51.883922 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:16:51.883922 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 14:16:51.883922 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:16:51.883922 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 652/1298] 10.222.0.52 () {56 vars in 1117 bytes} [Sat Sep 24 14:16:51 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:16:51.888451 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa094bda-b8c4-4483-8ad0-a75e9af97a7b None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:51.906441 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:51.988023 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa094bda-b8c4-4483-8ad0-a75e9af97a7b None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:52.021138 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c76a9d3-5421-435b-a7b5-026952736a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:52.021738 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76a9d3-5421-435b-a7b5-026952736a81 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:52.022015 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76a9d3-5421-435b-a7b5-026952736a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:52.022284 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76a9d3-5421-435b-a7b5-026952736a81 None admin] PATH_INFO: `/v3/services` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:52.022938 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c76a9d3-5421-435b-a7b5-026952736a81 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:16:52.023239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c76a9d3-5421-435b-a7b5-026952736a81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:52.024595 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c76a9d3-5421-435b-a7b5-026952736a81 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:52.029558 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 647/1299] 10.222.0.52 () {56 vars in 1116 bytes} [Sat Sep 24 14:16:51 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:16:52.035954 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e34f3ab-04d2-408c-8b0b-a7e619216cef None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:52.066990 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:52.132975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e34f3ab-04d2-408c-8b0b-a7e619216cef None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:52.165603 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff998d9-0137-4131-8845-16fc82fa3ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:52.166201 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff998d9-0137-4131-8845-16fc82fa3ddb None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:52.166480 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff998d9-0137-4131-8845-16fc82fa3ddb None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:52.166752 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff998d9-0137-4131-8845-16fc82fa3ddb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:52.167472 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff998d9-0137-4131-8845-16fc82fa3ddb None admin] RBAC: Adding query filter params (interface=public, service_id=95d2c84efd834ee4ba1396d8fdcddd1a, region_id=RegionOne) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:16:52.167780 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff998d9-0137-4131-8845-16fc82fa3ddb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:52.169141 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff998d9-0137-4131-8845-16fc82fa3ddb None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:52.173983 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 653/1300] 10.222.0.52 () {56 vars in 1251 bytes} [Sat Sep 24 14:16:52 2022] GET /identity/v3/endpoints?service_id=95d2c84efd834ee4ba1396d8fdcddd1a&interface=public®ion_id=RegionOne => generated 194 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:16:52.182253 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c76a9d3-5421-435b-a7b5-026952736a81 None admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:52.263493 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:52.281523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c76a9d3-5421-435b-a7b5-026952736a81 None admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:52.315115 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad269ff3-14b5-4c79-bbb7-e9c4f077cdb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:52.315711 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad269ff3-14b5-4c79-bbb7-e9c4f077cdb1 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:52.315990 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad269ff3-14b5-4c79-bbb7-e9c4f077cdb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:52.316259 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad269ff3-14b5-4c79-bbb7-e9c4f077cdb1 None admin] PATH_INFO: `/v3/services` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:52.316861 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad269ff3-14b5-4c79-bbb7-e9c4f077cdb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:52.318215 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad269ff3-14b5-4c79-bbb7-e9c4f077cdb1 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:52.324761 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 648/1301] 10.222.0.52 () {56 vars in 1087 bytes} [Sat Sep 24 14:16:52 2022] GET /identity/v3/services => generated 2191 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:16:54.440107 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe093d6b-ebc9-49fe-aae6-46adaf060ec9 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:54.440657 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe093d6b-ebc9-49fe-aae6-46adaf060ec9 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:54.440657 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe093d6b-ebc9-49fe-aae6-46adaf060ec9 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:54.441981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 654/1302] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:16:54.447265 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b8376a-4b36-4e5a-9401-2a581867d7fd None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:54.447687 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b8376a-4b36-4e5a-9401-2a581867d7fd None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:54.447814 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b8376a-4b36-4e5a-9401-2a581867d7fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:54.465387 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-c5b8376a-4b36-4e5a-9401-2a581867d7fd None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:16:54.466550 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-c5b8376a-4b36-4e5a-9401-2a581867d7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:16:54.530894 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 649/1303] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:16:54 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 14:16:54.798202 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe093d6b-ebc9-49fe-aae6-46adaf060ec9 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:54.894279 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:54.901319 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe093d6b-ebc9-49fe-aae6-46adaf060ec9 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:54.934303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b47b61d-b501-4d21-a598-b5248b4523e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:54.934959 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47b61d-b501-4d21-a598-b5248b4523e6 None admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:54.935241 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47b61d-b501-4d21-a598-b5248b4523e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:54.935509 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47b61d-b501-4d21-a598-b5248b4523e6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:54.936084 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b47b61d-b501-4d21-a598-b5248b4523e6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:54.937455 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b47b61d-b501-4d21-a598-b5248b4523e6 None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:54.947229 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application [None req-1b47b61d-b501-4d21-a598-b5248b4523e6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 14:16:54.947229 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:16:54.947229 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 14:16:54.947229 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 14:16:54.947229 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 14:16:54.947229 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 14:16:54.947229 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 14:16:54.947229 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:16:54.947229 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 14:16:54.947229 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:16:54.947229 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 14:16:54.947229 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:16:54.947229 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 14:16:54.947229 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 14:16:54.947229 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 14:16:54.947229 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 14:16:54.947229 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 14:16:54.948246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 14:16:54.948246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 14:16:54.948246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 14:16:54.948246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 14:16:54.948246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 14:16:54.948246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 14:16:54.948246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 14:16:54.948246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 14:16:54.948246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 14:16:54.948246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 14:16:54.948246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 14:16:54.948246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 14:16:54.948246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 14:16:54.948246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 14:16:54.948246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 14:16:54.948246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 14:16:54.949130 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 14:16:54.949130 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application with Lock( Sep 24 14:16:54.949130 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 14:16:54.949130 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self._enter() Sep 24 14:16:54.949130 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 14:16:54.949130 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 14:16:54.949130 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 14:16:54.949130 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application return self.creator() Sep 24 14:16:54.949130 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 14:16:54.949130 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application created_value = creator( Sep 24 14:16:54.949130 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 14:16:54.949130 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 14:16:54.949130 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 14:16:54.949130 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: ERROR keystone.server.flask.application Sep 24 14:16:54.949130 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 655/1304] 10.222.0.52 () {56 vars in 1117 bytes} [Sat Sep 24 14:16:54 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 14:16:54.954382 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b8376a-4b36-4e5a-9401-2a581867d7fd None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:55.028103 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:16:55.050644 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b8376a-4b36-4e5a-9401-2a581867d7fd None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:55.083470 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb64eac-d1f0-470e-886c-f659587b9534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:55.084050 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb64eac-d1f0-470e-886c-f659587b9534 None admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:55.084325 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb64eac-d1f0-470e-886c-f659587b9534 None admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:55.084592 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb64eac-d1f0-470e-886c-f659587b9534 None admin] PATH_INFO: `/v3/services` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:55.085209 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb64eac-d1f0-470e-886c-f659587b9534 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:16:55.085503 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb64eac-d1f0-470e-886c-f659587b9534 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:55.086870 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb64eac-d1f0-470e-886c-f659587b9534 None admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:55.091801 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 650/1305] 10.222.0.52 () {56 vars in 1116 bytes} [Sat Sep 24 14:16:54 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:16:55.097747 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b47b61d-b501-4d21-a598-b5248b4523e6 None admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:16:55.193847 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b47b61d-b501-4d21-a598-b5248b4523e6 None admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:16:55.227437 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6398dd-cbb2-4c15-adc8-96fdfc59110d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:16:55.228023 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6398dd-cbb2-4c15-adc8-96fdfc59110d None admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:16:55.228299 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6398dd-cbb2-4c15-adc8-96fdfc59110d None admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:16:55.228566 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6398dd-cbb2-4c15-adc8-96fdfc59110d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:16:55.229222 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6398dd-cbb2-4c15-adc8-96fdfc59110d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:16:55.230572 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6398dd-cbb2-4c15-adc8-96fdfc59110d None admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:16:55.249993 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 656/1306] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:16:55 2022] POST /identity/v3/endpoints => generated 331 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:17:05.088925 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=30 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:17:05.245734 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:17:59.038191 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793405b4-a18c-4040-aa94-aef4393b87d0 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:17:59.039465 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793405b4-a18c-4040-aa94-aef4393b87d0 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:17:59.040015 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793405b4-a18c-4040-aa94-aef4393b87d0 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:17:59.047907 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 651/1307] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:17:59 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:17:59.049524 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f465ac96-8f0a-4bc1-9e59-ba87e895cd15 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:17:59.049933 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f465ac96-8f0a-4bc1-9e59-ba87e895cd15 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:17:59.050540 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f465ac96-8f0a-4bc1-9e59-ba87e895cd15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:17:59.155489 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-f465ac96-8f0a-4bc1-9e59-ba87e895cd15 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:17:59.157094 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-f465ac96-8f0a-4bc1-9e59-ba87e895cd15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:17:59.274353 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 657/1308] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:17:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:17:59.569944 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74484e6-527e-493e-a961-294472e403a0 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:17:59.570221 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74484e6-527e-493e-a961-294472e403a0 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:17:59.570488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74484e6-527e-493e-a961-294472e403a0 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:17:59.572065 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 652/1309] 10.222.0.52 () {54 vars in 964 bytes} [Sat Sep 24 14:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:17:59.578466 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7600f08-09d2-4ffc-aabb-68172375e88f None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:17:59.578751 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7600f08-09d2-4ffc-aabb-68172375e88f None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:17:59.579069 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7600f08-09d2-4ffc-aabb-68172375e88f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:17:59.662863 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-f7600f08-09d2-4ffc-aabb-68172375e88f None None] MFA Rules not processed for user `a9348164132846d2b593ddfa29a820eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:17:59.664055 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-f7600f08-09d2-4ffc-aabb-68172375e88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:17:59.686008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:17:59.746267 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 658/1310] 10.222.0.52 () {58 vars in 1061 bytes} [Sat Sep 24 14:17:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:17:59.756297 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74484e6-527e-493e-a961-294472e403a0 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:17:59.758035 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-d74484e6-527e-493e-a961-294472e403a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:17:59.885255 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74484e6-527e-493e-a961-294472e403a0 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:17:59.967202 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abb28c9-9a53-4c2b-911e-6a36bbf86e84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:17:59.967896 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb28c9-9a53-4c2b-911e-6a36bbf86e84 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:17:59.968257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb28c9-9a53-4c2b-911e-6a36bbf86e84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:17:59.968564 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb28c9-9a53-4c2b-911e-6a36bbf86e84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:17:59.970237 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-2abb28c9-9a53-4c2b-911e-6a36bbf86e84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:00.029245 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:18:00.096612 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abb28c9-9a53-4c2b-911e-6a36bbf86e84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:18:00.098078 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abb28c9-9a53-4c2b-911e-6a36bbf86e84 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:18:00.118149 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 653/1311] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:17:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:18:02.573761 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b50d829-404b-4fdc-bded-ddc98c8866fd None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:02.574304 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b50d829-404b-4fdc-bded-ddc98c8866fd None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:02.574304 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b50d829-404b-4fdc-bded-ddc98c8866fd None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:02.575661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 659/1312] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:18:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:18:02.580817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddb8d9-a274-4601-80de-52794da31cdb None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:02.581259 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddb8d9-a274-4601-80de-52794da31cdb None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:02.581369 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddb8d9-a274-4601-80de-52794da31cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:02.661676 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-80ddb8d9-a274-4601-80de-52794da31cdb None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:18:02.662898 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-80ddb8d9-a274-4601-80de-52794da31cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:02.757305 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 654/1313] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:18:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:18:03.040723 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b50d829-404b-4fdc-bded-ddc98c8866fd None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:18:03.042249 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-9b50d829-404b-4fdc-bded-ddc98c8866fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:03.147472 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b50d829-404b-4fdc-bded-ddc98c8866fd None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:18:03.216925 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:18:03.220077 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0517c4-fadd-41a7-bad3-13f87a10accf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:18:03.220761 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0517c4-fadd-41a7-bad3-13f87a10accf service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:03.221284 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0517c4-fadd-41a7-bad3-13f87a10accf service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:03.221769 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0517c4-fadd-41a7-bad3-13f87a10accf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:03.223626 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-7b0517c4-fadd-41a7-bad3-13f87a10accf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:03.333135 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0517c4-fadd-41a7-bad3-13f87a10accf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:18:03.334581 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0517c4-fadd-41a7-bad3-13f87a10accf service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:18:03.352692 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 660/1314] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:18:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:18:05.858420 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af3c363-d32c-46f7-9bae-ec8c620b2a0d None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:05.858960 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af3c363-d32c-46f7-9bae-ec8c620b2a0d None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:05.859070 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af3c363-d32c-46f7-9bae-ec8c620b2a0d None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:05.860340 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 655/1315] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:18:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:18:05.865767 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ddfd5-5d99-4943-85e9-bbd99c2b69e3 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:05.866194 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ddfd5-5d99-4943-85e9-bbd99c2b69e3 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:05.866345 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ddfd5-5d99-4943-85e9-bbd99c2b69e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:05.933598 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-988ddfd5-5d99-4943-85e9-bbd99c2b69e3 None None] MFA Rules not processed for user `75510b26c21742e091c6eca468df1c06`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:18:05.934789 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-988ddfd5-5d99-4943-85e9-bbd99c2b69e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:05.985341 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:18:06.030467 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 661/1316] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:18:05 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:18:06.351433 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f7f65-2507-40ac-8f16-9e5791a0da48 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:06.351715 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f7f65-2507-40ac-8f16-9e5791a0da48 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:06.351993 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f7f65-2507-40ac-8f16-9e5791a0da48 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:06.353351 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 656/1317] 10.222.0.52 () {54 vars in 964 bytes} [Sat Sep 24 14:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:18:06.359850 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad20313-4c5a-4c37-a4a5-79c8953c9d8c None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:06.360134 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad20313-4c5a-4c37-a4a5-79c8953c9d8c None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:06.360401 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad20313-4c5a-4c37-a4a5-79c8953c9d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:06.436289 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-6ad20313-4c5a-4c37-a4a5-79c8953c9d8c None None] MFA Rules not processed for user `a9348164132846d2b593ddfa29a820eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:18:06.437488 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-6ad20313-4c5a-4c37-a4a5-79c8953c9d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:06.516358 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 662/1318] 10.222.0.52 () {58 vars in 1061 bytes} [Sat Sep 24 14:18:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:18:06.525492 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f7f65-2507-40ac-8f16-9e5791a0da48 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:18:06.527100 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-191f7f65-2507-40ac-8f16-9e5791a0da48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:06.537700 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:18:06.629179 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f7f65-2507-40ac-8f16-9e5791a0da48 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:18:06.703431 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6360bc8-602d-4059-a4c1-bfa847dd0430 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:18:06.704038 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6360bc8-602d-4059-a4c1-bfa847dd0430 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:06.704337 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6360bc8-602d-4059-a4c1-bfa847dd0430 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:06.704628 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6360bc8-602d-4059-a4c1-bfa847dd0430 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:06.706211 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-b6360bc8-602d-4059-a4c1-bfa847dd0430 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:06.767689 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:18:06.821036 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6360bc8-602d-4059-a4c1-bfa847dd0430 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:18:06.822512 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6360bc8-602d-4059-a4c1-bfa847dd0430 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:18:06.840205 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 657/1319] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:18:06 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:18:09.886217 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661576d9-403b-4366-ae4d-aa6cc62711ae None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:09.886702 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661576d9-403b-4366-ae4d-aa6cc62711ae None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:09.886846 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661576d9-403b-4366-ae4d-aa6cc62711ae None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:09.888152 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 663/1320] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:18:09.893854 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464aaa6c-a7bb-4b3b-a9a6-cbfd44253b95 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:09.894256 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464aaa6c-a7bb-4b3b-a9a6-cbfd44253b95 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:09.894402 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464aaa6c-a7bb-4b3b-a9a6-cbfd44253b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:09.959093 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-464aaa6c-a7bb-4b3b-a9a6-cbfd44253b95 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:18:09.960292 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-464aaa6c-a7bb-4b3b-a9a6-cbfd44253b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:10.054142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 658/1321] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:18:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:18:10.335203 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661576d9-403b-4366-ae4d-aa6cc62711ae None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:18:10.336738 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-661576d9-403b-4366-ae4d-aa6cc62711ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:10.386604 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:18:10.440165 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661576d9-403b-4366-ae4d-aa6cc62711ae None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:18:10.512834 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b59bf-057c-4efa-bd8e-2e8c4d2fe9e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:18:10.513391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b59bf-057c-4efa-bd8e-2e8c4d2fe9e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:10.513670 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b59bf-057c-4efa-bd8e-2e8c4d2fe9e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:10.514461 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b59bf-057c-4efa-bd8e-2e8c4d2fe9e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:10.516056 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-8a0b59bf-057c-4efa-bd8e-2e8c4d2fe9e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:10.611467 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:18:10.624817 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0b59bf-057c-4efa-bd8e-2e8c4d2fe9e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:18:10.626249 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0b59bf-057c-4efa-bd8e-2e8c4d2fe9e1 service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:18:10.644079 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 664/1322] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:18:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:18:13.484240 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1168cc50-d2e8-47cd-9fde-dc4a5f0c9be6 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:13.484790 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1168cc50-d2e8-47cd-9fde-dc4a5f0c9be6 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:13.484790 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1168cc50-d2e8-47cd-9fde-dc4a5f0c9be6 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:13.486703 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 659/1323] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:18:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:18:13.491925 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612cdd7-eb9c-483e-84e9-28ebeac2bbaa None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:13.492461 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612cdd7-eb9c-483e-84e9-28ebeac2bbaa None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:13.492461 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612cdd7-eb9c-483e-84e9-28ebeac2bbaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:13.567215 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-3612cdd7-eb9c-483e-84e9-28ebeac2bbaa None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:18:13.568682 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-3612cdd7-eb9c-483e-84e9-28ebeac2bbaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:13.673381 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 665/1324] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:18:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:18:13.955621 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1168cc50-d2e8-47cd-9fde-dc4a5f0c9be6 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:18:13.957150 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-1168cc50-d2e8-47cd-9fde-dc4a5f0c9be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:13.977088 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:18:14.061066 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1168cc50-d2e8-47cd-9fde-dc4a5f0c9be6 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:18:14.134033 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e1f8d1-254c-45d0-95b8-696f847796ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:18:14.134990 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1f8d1-254c-45d0-95b8-696f847796ec service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:14.135274 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1f8d1-254c-45d0-95b8-696f847796ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:14.135548 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1f8d1-254c-45d0-95b8-696f847796ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:14.137139 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-41e1f8d1-254c-45d0-95b8-696f847796ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:14.210326 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:18:14.251326 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e1f8d1-254c-45d0-95b8-696f847796ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:18:14.255747 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e1f8d1-254c-45d0-95b8-696f847796ec service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:18:14.275385 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 660/1325] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:18:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:18:16.480425 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cf671c-6f05-4ef0-9bbb-8e8cb3cdc662 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:16.480959 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cf671c-6f05-4ef0-9bbb-8e8cb3cdc662 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:16.480959 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cf671c-6f05-4ef0-9bbb-8e8cb3cdc662 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:16.482854 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 666/1326] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:18:16.488049 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a9c5d-0260-4636-a399-53b6414fdf8a None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:16.488507 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a9c5d-0260-4636-a399-53b6414fdf8a None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:16.488604 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a9c5d-0260-4636-a399-53b6414fdf8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:16.565865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-4b9a9c5d-0260-4636-a399-53b6414fdf8a None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:18:16.567106 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-4b9a9c5d-0260-4636-a399-53b6414fdf8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:16.664328 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 661/1327] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:18:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:18:16.947157 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cf671c-6f05-4ef0-9bbb-8e8cb3cdc662 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:18:16.948666 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-85cf671c-6f05-4ef0-9bbb-8e8cb3cdc662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:17.020546 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:18:17.064125 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cf671c-6f05-4ef0-9bbb-8e8cb3cdc662 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:18:17.137200 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993baa03-35d5-4644-8b36-529b0b167783 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:18:17.137807 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993baa03-35d5-4644-8b36-529b0b167783 service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:17.138084 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993baa03-35d5-4644-8b36-529b0b167783 service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:17.138348 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993baa03-35d5-4644-8b36-529b0b167783 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:17.140454 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-993baa03-35d5-4644-8b36-529b0b167783 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:17.247879 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:18:17.250590 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993baa03-35d5-4644-8b36-529b0b167783 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:18:17.252692 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993baa03-35d5-4644-8b36-529b0b167783 service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:18:17.271076 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 667/1328] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:18:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:18:19.476413 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53495405-ea04-4dd7-bebc-9d90cf390946 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:19.476904 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53495405-ea04-4dd7-bebc-9d90cf390946 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:19.476904 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53495405-ea04-4dd7-bebc-9d90cf390946 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:19.478231 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 662/1329] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:18:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:18:19.483463 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf61c0c-85ce-4819-bd38-5cde06132207 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:19.483885 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf61c0c-85ce-4819-bd38-5cde06132207 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:19.484070 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf61c0c-85ce-4819-bd38-5cde06132207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:19.549552 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-faf61c0c-85ce-4819-bd38-5cde06132207 None None] MFA Rules not processed for user `75510b26c21742e091c6eca468df1c06`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:18:19.550762 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-faf61c0c-85ce-4819-bd38-5cde06132207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:19.650402 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 668/1330] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:18:19 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:18:19.933006 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53495405-ea04-4dd7-bebc-9d90cf390946 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:18:19.934636 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-53495405-ea04-4dd7-bebc-9d90cf390946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:19.969197 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:18:20.045707 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53495405-ea04-4dd7-bebc-9d90cf390946 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:18:20.118773 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac6f7f-814c-49ff-bdbb-eb30beba7455 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:18:20.119396 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac6f7f-814c-49ff-bdbb-eb30beba7455 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:20.119669 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac6f7f-814c-49ff-bdbb-eb30beba7455 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:20.119934 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac6f7f-814c-49ff-bdbb-eb30beba7455 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:20.121511 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-25ac6f7f-814c-49ff-bdbb-eb30beba7455 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:20.197915 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:18:20.231324 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac6f7f-814c-49ff-bdbb-eb30beba7455 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:18:20.232827 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac6f7f-814c-49ff-bdbb-eb30beba7455 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:18:20.250923 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 663/1331] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:18:19 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:18:22.674578 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aceeacf-f4bf-41e1-995a-9e6aae8e2e15 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:22.675243 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aceeacf-f4bf-41e1-995a-9e6aae8e2e15 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:22.675243 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aceeacf-f4bf-41e1-995a-9e6aae8e2e15 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:22.676507 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 669/1332] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:18:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:18:22.681717 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ca923-1aab-4ba3-b507-29184a4ba49c None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:22.682109 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ca923-1aab-4ba3-b507-29184a4ba49c None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:22.682257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ca923-1aab-4ba3-b507-29184a4ba49c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:22.746606 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-c71ca923-1aab-4ba3-b507-29184a4ba49c None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:18:22.748381 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-c71ca923-1aab-4ba3-b507-29184a4ba49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:22.841269 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 664/1333] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:18:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:18:23.119350 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aceeacf-f4bf-41e1-995a-9e6aae8e2e15 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:18:23.120876 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-1aceeacf-f4bf-41e1-995a-9e6aae8e2e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:23.188850 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:18:23.224355 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aceeacf-f4bf-41e1-995a-9e6aae8e2e15 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:18:23.296091 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7941c4be-9e23-450e-a515-129b6ebbb821 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:18:23.296685 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7941c4be-9e23-450e-a515-129b6ebbb821 service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:23.296957 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7941c4be-9e23-450e-a515-129b6ebbb821 service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:23.297221 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7941c4be-9e23-450e-a515-129b6ebbb821 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:23.298776 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-7941c4be-9e23-450e-a515-129b6ebbb821 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:23.403829 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7941c4be-9e23-450e-a515-129b6ebbb821 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:18:23.405756 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7941c4be-9e23-450e-a515-129b6ebbb821 service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:18:23.415609 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:18:23.424124 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 670/1334] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:18:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:18:25.595138 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433daef-776c-4ffe-9271-b5128a1f0303 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:25.595780 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433daef-776c-4ffe-9271-b5128a1f0303 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:25.595780 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433daef-776c-4ffe-9271-b5128a1f0303 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:25.597009 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 665/1335] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:18:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:18:25.602106 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf756166-b113-4c13-b44b-966674f8edd7 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:25.602543 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf756166-b113-4c13-b44b-966674f8edd7 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:25.602639 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf756166-b113-4c13-b44b-966674f8edd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:25.668763 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-bf756166-b113-4c13-b44b-966674f8edd7 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:18:25.670491 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-bf756166-b113-4c13-b44b-966674f8edd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:25.764304 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 671/1336] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:18:26.049020 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433daef-776c-4ffe-9271-b5128a1f0303 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:18:26.050545 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-2433daef-776c-4ffe-9271-b5128a1f0303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:26.089645 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:18:26.151888 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433daef-776c-4ffe-9271-b5128a1f0303 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:18:26.222545 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07a1bb-8d10-4b16-9b3d-ab00ebacc732 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:18:26.223165 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07a1bb-8d10-4b16-9b3d-ab00ebacc732 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:26.223441 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07a1bb-8d10-4b16-9b3d-ab00ebacc732 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:26.223707 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07a1bb-8d10-4b16-9b3d-ab00ebacc732 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:26.225262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-ef07a1bb-8d10-4b16-9b3d-ab00ebacc732 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:26.308578 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:18:26.333953 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07a1bb-8d10-4b16-9b3d-ab00ebacc732 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:18:26.335421 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07a1bb-8d10-4b16-9b3d-ab00ebacc732 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:18:26.353524 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 666/1337] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:18:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:18:28.960593 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69d527-4835-4606-bab1-090632f48915 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:28.961180 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69d527-4835-4606-bab1-090632f48915 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:28.961180 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69d527-4835-4606-bab1-090632f48915 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:28.962461 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 672/1338] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:18:28.968025 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b327df-00f5-4231-b78d-a5a3e57d8ea6 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:28.968477 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b327df-00f5-4231-b78d-a5a3e57d8ea6 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:28.968580 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b327df-00f5-4231-b78d-a5a3e57d8ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:29.033510 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-e0b327df-00f5-4231-b78d-a5a3e57d8ea6 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:18:29.034689 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-e0b327df-00f5-4231-b78d-a5a3e57d8ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:29.131106 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 667/1339] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:18:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:18:29.410673 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd69d527-4835-4606-bab1-090632f48915 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:18:29.412255 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-dd69d527-4835-4606-bab1-090632f48915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:29.486685 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:18:29.513501 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd69d527-4835-4606-bab1-090632f48915 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:18:29.587224 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5b84b6-3b1d-457d-89e7-12c1853dc013 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:18:29.587825 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b84b6-3b1d-457d-89e7-12c1853dc013 service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:29.588100 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b84b6-3b1d-457d-89e7-12c1853dc013 service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:29.588367 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b84b6-3b1d-457d-89e7-12c1853dc013 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:29.589936 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-0f5b84b6-3b1d-457d-89e7-12c1853dc013 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:29.695276 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5b84b6-3b1d-457d-89e7-12c1853dc013 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:18:29.696645 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5b84b6-3b1d-457d-89e7-12c1853dc013 service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:18:29.714574 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 673/1340] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:18:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:18:33.838342 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a909f99-9762-41b3-a061-28aa0f2d1d47 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:33.838782 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a909f99-9762-41b3-a061-28aa0f2d1d47 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:33.838993 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a909f99-9762-41b3-a061-28aa0f2d1d47 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:33.840422 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 668/1341] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:18:33.846497 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ec2d1-d828-43bf-b902-923277f073ec None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:33.847016 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ec2d1-d828-43bf-b902-923277f073ec None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:33.847112 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ec2d1-d828-43bf-b902-923277f073ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:33.852699 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:18:33.920660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-ca5ec2d1-d828-43bf-b902-923277f073ec None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:18:33.921901 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-ca5ec2d1-d828-43bf-b902-923277f073ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:34.026775 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 674/1342] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:18:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:18:34.315510 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a909f99-9762-41b3-a061-28aa0f2d1d47 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:18:34.317116 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-0a909f99-9762-41b3-a061-28aa0f2d1d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:34.368663 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:18:34.423821 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a909f99-9762-41b3-a061-28aa0f2d1d47 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:18:34.497800 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6567f0b3-7586-483b-a501-a6738af83d49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:18:34.498405 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6567f0b3-7586-483b-a501-a6738af83d49 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:34.498678 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6567f0b3-7586-483b-a501-a6738af83d49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:34.498982 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6567f0b3-7586-483b-a501-a6738af83d49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:34.500550 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-6567f0b3-7586-483b-a501-a6738af83d49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:34.594609 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:18:34.611515 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6567f0b3-7586-483b-a501-a6738af83d49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:18:34.612951 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6567f0b3-7586-483b-a501-a6738af83d49 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:18:34.633717 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 669/1343] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:18:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:18:37.425006 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3134c99-ef9b-4b60-ba89-515e4870f0c2 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:37.425542 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3134c99-ef9b-4b60-ba89-515e4870f0c2 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:37.425542 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3134c99-ef9b-4b60-ba89-515e4870f0c2 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:37.426914 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 675/1344] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:18:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:18:37.431954 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8285642-0a10-4e1a-99c3-d543cef90fa8 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:37.432389 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8285642-0a10-4e1a-99c3-d543cef90fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:37.432483 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8285642-0a10-4e1a-99c3-d543cef90fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:37.497365 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-c8285642-0a10-4e1a-99c3-d543cef90fa8 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:18:37.498574 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-c8285642-0a10-4e1a-99c3-d543cef90fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:37.601216 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 670/1345] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:18:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:18:37.887134 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3134c99-ef9b-4b60-ba89-515e4870f0c2 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:18:37.889291 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-a3134c99-ef9b-4b60-ba89-515e4870f0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:37.978159 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:18:37.998430 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3134c99-ef9b-4b60-ba89-515e4870f0c2 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:18:38.076712 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a11f7-a39b-42c0-9103-6f26d90b85d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:18:38.077312 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a11f7-a39b-42c0-9103-6f26d90b85d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:38.077587 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a11f7-a39b-42c0-9103-6f26d90b85d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:38.077854 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a11f7-a39b-42c0-9103-6f26d90b85d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:38.079460 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-093a11f7-a39b-42c0-9103-6f26d90b85d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:38.187708 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093a11f7-a39b-42c0-9103-6f26d90b85d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:18:38.189151 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093a11f7-a39b-42c0-9103-6f26d90b85d9 service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:18:38.207207 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 676/1346] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:18:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:18:42.417883 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0ab61-16a1-4960-8cac-91fd8a41d427 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:42.418407 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0ab61-16a1-4960-8cac-91fd8a41d427 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:42.419050 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0ab61-16a1-4960-8cac-91fd8a41d427 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:42.420438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 671/1347] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:18:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:18:42.425833 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf8791-9559-4262-9047-4babbbb7d095 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:42.426252 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf8791-9559-4262-9047-4babbbb7d095 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:42.426376 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf8791-9559-4262-9047-4babbbb7d095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:42.443519 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:18:42.503216 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-dacf8791-9559-4262-9047-4babbbb7d095 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:18:42.504537 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-dacf8791-9559-4262-9047-4babbbb7d095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:42.616088 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 677/1348] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:18:42.896453 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e0ab61-16a1-4960-8cac-91fd8a41d427 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:18:42.897988 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-32e0ab61-16a1-4960-8cac-91fd8a41d427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:42.955556 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:18:43.000552 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e0ab61-16a1-4960-8cac-91fd8a41d427 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:18:43.071983 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300bb103-7e56-4b64-a262-65e36fd0d8a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:18:43.072576 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300bb103-7e56-4b64-a262-65e36fd0d8a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:43.072851 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300bb103-7e56-4b64-a262-65e36fd0d8a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:43.073116 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300bb103-7e56-4b64-a262-65e36fd0d8a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:43.075265 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-300bb103-7e56-4b64-a262-65e36fd0d8a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:43.179463 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:18:43.183759 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300bb103-7e56-4b64-a262-65e36fd0d8a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:18:43.185717 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300bb103-7e56-4b64-a262-65e36fd0d8a1 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:18:43.204071 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 672/1349] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:18:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:18:48.110221 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a6e29-221b-4473-9f8c-4abf6ed42d48 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:48.110645 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a6e29-221b-4473-9f8c-4abf6ed42d48 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:48.110789 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a6e29-221b-4473-9f8c-4abf6ed42d48 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:48.123367 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 678/1350] 10.222.0.52 () {54 vars in 921 bytes} [Sat Sep 24 14:18:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:18:48.125330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b45656-f79f-496e-b1ec-0e08fb6d19db None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:48.126197 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b45656-f79f-496e-b1ec-0e08fb6d19db None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:48.126618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b45656-f79f-496e-b1ec-0e08fb6d19db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:48.252984 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-c6b45656-f79f-496e-b1ec-0e08fb6d19db None None] MFA Rules not processed for user `18fe384ecd454fb298dcc788f41dd44b`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:18:48.254575 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-c6b45656-f79f-496e-b1ec-0e08fb6d19db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:48.376612 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 673/1351] 10.222.0.52 () {58 vars in 1018 bytes} [Sat Sep 24 14:18:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:18:52.557122 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6b655-37d5-4940-b4a3-09398cc8b7d9 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:52.557549 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6b655-37d5-4940-b4a3-09398cc8b7d9 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:52.557650 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6b655-37d5-4940-b4a3-09398cc8b7d9 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:52.559801 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 679/1352] 10.222.0.52 () {54 vars in 921 bytes} [Sat Sep 24 14:18:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:18:52.564507 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd5971-e464-4fa9-ac85-03b8ccfe3531 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:52.564924 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd5971-e464-4fa9-ac85-03b8ccfe3531 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:52.565062 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd5971-e464-4fa9-ac85-03b8ccfe3531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:52.614464 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=59 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:18:52.623163 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:18:52.646881 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-f1dd5971-e464-4fa9-ac85-03b8ccfe3531 None None] MFA Rules not processed for user `18fe384ecd454fb298dcc788f41dd44b`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:18:52.648086 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-f1dd5971-e464-4fa9-ac85-03b8ccfe3531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:52.767169 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 674/1353] 10.222.0.52 () {58 vars in 1018 bytes} [Sat Sep 24 14:18:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:18:53.996758 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e417d-0622-409e-b20e-dbcbb4fd8a0d None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:53.997293 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e417d-0622-409e-b20e-dbcbb4fd8a0d None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:53.997293 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e417d-0622-409e-b20e-dbcbb4fd8a0d None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:53.998644 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 680/1354] 10.222.0.52 () {54 vars in 921 bytes} [Sat Sep 24 14:18:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:18:54.003643 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d531e339-b181-409f-a396-1e259f2f3554 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:54.004067 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d531e339-b181-409f-a396-1e259f2f3554 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:54.004181 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d531e339-b181-409f-a396-1e259f2f3554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:54.080677 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-d531e339-b181-409f-a396-1e259f2f3554 None None] MFA Rules not processed for user `18fe384ecd454fb298dcc788f41dd44b`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:18:54.081878 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-d531e339-b181-409f-a396-1e259f2f3554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:54.123913 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:18:54.189434 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 675/1355] 10.222.0.52 () {58 vars in 1018 bytes} [Sat Sep 24 14:18:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:18:57.901738 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd949695-d758-453a-af5c-504db8a53491 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:57.902868 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd949695-d758-453a-af5c-504db8a53491 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:57.902868 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd949695-d758-453a-af5c-504db8a53491 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:57.904479 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 681/1356] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:18:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:18:57.909964 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b12ab9-141c-4c64-9d2b-35b84d1945be None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:57.910765 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b12ab9-141c-4c64-9d2b-35b84d1945be None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:57.911079 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b12ab9-141c-4c64-9d2b-35b84d1945be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:57.986510 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-70b12ab9-141c-4c64-9d2b-35b84d1945be None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:18:57.987665 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-70b12ab9-141c-4c64-9d2b-35b84d1945be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:58.098865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 676/1357] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:18:58.400328 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61faa6c6-77be-4da5-a1a7-d056fe81d48f None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:58.401064 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61faa6c6-77be-4da5-a1a7-d056fe81d48f None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:58.401569 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61faa6c6-77be-4da5-a1a7-d056fe81d48f None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:58.403675 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 682/1358] 10.222.0.52 () {54 vars in 961 bytes} [Sat Sep 24 14:18:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:18:58.416690 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d150c-e3e6-4dfb-a59f-262709208687 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:58.417262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d150c-e3e6-4dfb-a59f-262709208687 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:58.418513 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d150c-e3e6-4dfb-a59f-262709208687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:58.442349 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:18:58.505189 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-217d150c-e3e6-4dfb-a59f-262709208687 None None] MFA Rules not processed for user `79eaaae97aac4282a613939e5b768376`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:18:58.506370 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-217d150c-e3e6-4dfb-a59f-262709208687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:58.632902 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 677/1359] 10.222.0.52 () {58 vars in 1058 bytes} [Sat Sep 24 14:18:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:18:58.641056 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61faa6c6-77be-4da5-a1a7-d056fe81d48f None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:18:58.642558 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-61faa6c6-77be-4da5-a1a7-d056fe81d48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:58.796089 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61faa6c6-77be-4da5-a1a7-d056fe81d48f None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:18:58.880949 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:18:58.923896 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f70e2a-a5af-4734-bfe9-1fa2a8f85d09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:18:58.924673 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f70e2a-a5af-4734-bfe9-1fa2a8f85d09 service nova] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:18:58.924910 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f70e2a-a5af-4734-bfe9-1fa2a8f85d09 service nova] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:18:58.925181 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f70e2a-a5af-4734-bfe9-1fa2a8f85d09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:18:58.926775 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-54f70e2a-a5af-4734-bfe9-1fa2a8f85d09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:18:59.027321 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f70e2a-a5af-4734-bfe9-1fa2a8f85d09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:18:59.028952 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f70e2a-a5af-4734-bfe9-1fa2a8f85d09 service nova] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:18:59.048621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 683/1360] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:18:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:19:01.445526 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323c47a-cab4-4636-b39f-9fa30da29f04 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:01.446014 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323c47a-cab4-4636-b39f-9fa30da29f04 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:01.446503 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323c47a-cab4-4636-b39f-9fa30da29f04 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:01.447457 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 678/1361] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:19:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:19:01.453221 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a05ad-f7d8-4e1c-b0ab-4ac96804a8ac None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:01.453628 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a05ad-f7d8-4e1c-b0ab-4ac96804a8ac None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:01.453871 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a05ad-f7d8-4e1c-b0ab-4ac96804a8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:01.522976 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-6e6a05ad-f7d8-4e1c-b0ab-4ac96804a8ac None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:19:01.524181 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-6e6a05ad-f7d8-4e1c-b0ab-4ac96804a8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:01.564809 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:01.627367 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 684/1362] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:19:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:19:01.915630 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323c47a-cab4-4636-b39f-9fa30da29f04 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:19:01.918027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-c323c47a-cab4-4636-b39f-9fa30da29f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:02.001267 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:02.111242 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323c47a-cab4-4636-b39f-9fa30da29f04 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:19:02.254513 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cb3321-20fa-4f9b-9b58-d84dccee2798 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:19:02.255369 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb3321-20fa-4f9b-9b58-d84dccee2798 service nova] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:02.255640 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb3321-20fa-4f9b-9b58-d84dccee2798 service nova] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:02.255910 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb3321-20fa-4f9b-9b58-d84dccee2798 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:02.257513 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-26cb3321-20fa-4f9b-9b58-d84dccee2798 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:02.283111 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:02.355164 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cb3321-20fa-4f9b-9b58-d84dccee2798 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:19:02.356781 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cb3321-20fa-4f9b-9b58-d84dccee2798 service nova] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:19:02.375365 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 679/1363] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:19:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:19:04.884706 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256bfd6-9e68-4fc6-aef4-9ef55d6edb63 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:04.885315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256bfd6-9e68-4fc6-aef4-9ef55d6edb63 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:04.885315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256bfd6-9e68-4fc6-aef4-9ef55d6edb63 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:04.886645 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 685/1364] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:19:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:19:04.891910 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa10f9-d857-4a8a-9494-055887629ab7 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:04.892312 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa10f9-d857-4a8a-9494-055887629ab7 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:04.892476 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa10f9-d857-4a8a-9494-055887629ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:04.960075 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-8baa10f9-d857-4a8a-9494-055887629ab7 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:19:04.961268 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-8baa10f9-d857-4a8a-9494-055887629ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:05.067110 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:05.078605 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 680/1365] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:19:05.365017 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d484d6a-11da-45d2-9ddd-872e79207d20 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:05.365405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d484d6a-11da-45d2-9ddd-872e79207d20 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:05.365776 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d484d6a-11da-45d2-9ddd-872e79207d20 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:05.367383 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 686/1366] 10.222.0.52 () {54 vars in 961 bytes} [Sat Sep 24 14:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:19:05.373234 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c138b9-57af-461a-a6d3-8bf274a175e4 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:05.373866 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c138b9-57af-461a-a6d3-8bf274a175e4 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:05.374148 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c138b9-57af-461a-a6d3-8bf274a175e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:05.452763 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-18c138b9-57af-461a-a6d3-8bf274a175e4 None None] MFA Rules not processed for user `79eaaae97aac4282a613939e5b768376`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:19:05.454135 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-18c138b9-57af-461a-a6d3-8bf274a175e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:05.577005 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 681/1367] 10.222.0.52 () {58 vars in 1058 bytes} [Sat Sep 24 14:19:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:19:05.585550 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d484d6a-11da-45d2-9ddd-872e79207d20 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:19:05.587167 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-5d484d6a-11da-45d2-9ddd-872e79207d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:05.748451 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d484d6a-11da-45d2-9ddd-872e79207d20 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:19:05.770577 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:05.876096 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180e10f-5467-43d3-8787-8b377556604d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:19:05.876854 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180e10f-5467-43d3-8787-8b377556604d service nova] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:05.877245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180e10f-5467-43d3-8787-8b377556604d service nova] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:05.877610 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180e10f-5467-43d3-8787-8b377556604d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:05.879303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-3180e10f-5467-43d3-8787-8b377556604d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:05.982116 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3180e10f-5467-43d3-8787-8b377556604d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:19:05.984024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3180e10f-5467-43d3-8787-8b377556604d service nova] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:19:06.003810 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 687/1368] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:19:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:19:06.060511 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982182e2-7c2c-4aba-a741-2a97023c2afd None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:06.060982 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982182e2-7c2c-4aba-a741-2a97023c2afd None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:06.061306 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982182e2-7c2c-4aba-a741-2a97023c2afd None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:06.062845 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 682/1369] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:19:06.068153 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b2265-3d56-4533-9e20-1fa919883eeb None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:06.068635 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b2265-3d56-4533-9e20-1fa919883eeb None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:06.069051 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b2265-3d56-4533-9e20-1fa919883eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:06.104287 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:06.148543 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-257b2265-3d56-4533-9e20-1fa919883eeb None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:19:06.149759 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-257b2265-3d56-4533-9e20-1fa919883eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:06.284989 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 688/1370] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:19:06.575770 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8dc97-5b05-4077-bc59-31a1f1c637a3 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:06.576071 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8dc97-5b05-4077-bc59-31a1f1c637a3 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:06.576328 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8dc97-5b05-4077-bc59-31a1f1c637a3 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:06.577949 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 683/1371] 10.222.0.52 () {54 vars in 963 bytes} [Sat Sep 24 14:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:19:06.582768 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e498fea-86f6-45be-a214-8de4e565c2d4 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:06.583089 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e498fea-86f6-45be-a214-8de4e565c2d4 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:06.583364 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e498fea-86f6-45be-a214-8de4e565c2d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:06.664112 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-5e498fea-86f6-45be-a214-8de4e565c2d4 None None] MFA Rules not processed for user `63921623056045a9b97b2de3ca418138`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:19:06.665314 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-5e498fea-86f6-45be-a214-8de4e565c2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:06.713057 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:06.749343 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 689/1372] 10.222.0.52 () {58 vars in 1060 bytes} [Sat Sep 24 14:19:06 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:19:06.757190 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c8dc97-5b05-4077-bc59-31a1f1c637a3 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:19:06.759139 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-04c8dc97-5b05-4077-bc59-31a1f1c637a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:06.818800 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:06.881077 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c8dc97-5b05-4077-bc59-31a1f1c637a3 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:19:06.960846 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e1ee86-e946-4487-b4b1-6932d77cd217 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:19:06.961457 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e1ee86-e946-4487-b4b1-6932d77cd217 service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:06.961732 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e1ee86-e946-4487-b4b1-6932d77cd217 service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:06.961993 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e1ee86-e946-4487-b4b1-6932d77cd217 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:06.963634 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-73e1ee86-e946-4487-b4b1-6932d77cd217 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:07.070141 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:07.098176 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e1ee86-e946-4487-b4b1-6932d77cd217 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:19:07.100196 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e1ee86-e946-4487-b4b1-6932d77cd217 service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:19:07.124959 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 684/1373] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:19:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:19:08.283046 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098e58a-9db1-4026-ad26-407fc13a2e91 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:08.283626 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098e58a-9db1-4026-ad26-407fc13a2e91 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:08.283626 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098e58a-9db1-4026-ad26-407fc13a2e91 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:08.284932 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 690/1374] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:19:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:19:08.290242 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ddc366-a2dd-48c4-a2a5-5aaf5789a199 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:08.290722 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ddc366-a2dd-48c4-a2a5-5aaf5789a199 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:08.290954 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ddc366-a2dd-48c4-a2a5-5aaf5789a199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:08.358831 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-77ddc366-a2dd-48c4-a2a5-5aaf5789a199 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:19:08.360047 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-77ddc366-a2dd-48c4-a2a5-5aaf5789a199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:08.458715 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 685/1375] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:19:08.763667 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3098e58a-9db1-4026-ad26-407fc13a2e91 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:19:08.765335 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-3098e58a-9db1-4026-ad26-407fc13a2e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:08.937740 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3098e58a-9db1-4026-ad26-407fc13a2e91 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:19:08.990972 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:09.054657 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29de49-2207-45b7-a291-e8fc5a1b4a41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:19:09.055323 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29de49-2207-45b7-a291-e8fc5a1b4a41 service nova] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:09.055602 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29de49-2207-45b7-a291-e8fc5a1b4a41 service nova] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:09.055873 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29de49-2207-45b7-a291-e8fc5a1b4a41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:09.057469 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-fb29de49-2207-45b7-a291-e8fc5a1b4a41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:09.152127 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29de49-2207-45b7-a291-e8fc5a1b4a41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:19:09.153796 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29de49-2207-45b7-a291-e8fc5a1b4a41 service nova] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:19:09.172605 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 691/1376] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:19:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:19:11.412865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d903799-a68d-4951-b888-70350e170e95 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:11.413394 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d903799-a68d-4951-b888-70350e170e95 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:11.413394 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d903799-a68d-4951-b888-70350e170e95 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:11.414792 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 686/1377] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:19:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:19:11.420199 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f224843-b565-48e3-b705-aa1559f9757e None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:11.420633 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f224843-b565-48e3-b705-aa1559f9757e None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:11.420964 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f224843-b565-48e3-b705-aa1559f9757e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:11.492840 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-9f224843-b565-48e3-b705-aa1559f9757e None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:19:11.494205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-9f224843-b565-48e3-b705-aa1559f9757e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:11.501856 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:11.607018 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 692/1378] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:19:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:19:11.887062 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d903799-a68d-4951-b888-70350e170e95 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:19:11.888879 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-3d903799-a68d-4951-b888-70350e170e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:11.951324 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:12.040922 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d903799-a68d-4951-b888-70350e170e95 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:19:12.158619 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172647a6-ce9a-4f37-9f58-785480c8a134 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:19:12.159349 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172647a6-ce9a-4f37-9f58-785480c8a134 service nova] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:12.159694 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172647a6-ce9a-4f37-9f58-785480c8a134 service nova] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:12.160032 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172647a6-ce9a-4f37-9f58-785480c8a134 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:12.161669 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-172647a6-ce9a-4f37-9f58-785480c8a134 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:12.181561 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:12.258067 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172647a6-ce9a-4f37-9f58-785480c8a134 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:19:12.259799 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172647a6-ce9a-4f37-9f58-785480c8a134 service nova] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:19:12.279111 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 687/1379] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:19:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:19:14.492843 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2fa67-67d3-4527-b199-c88121472417 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:14.493410 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2fa67-67d3-4527-b199-c88121472417 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:14.493410 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2fa67-67d3-4527-b199-c88121472417 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:14.495378 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 693/1380] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:19:14.501088 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298a112-ffee-4983-b5c4-96290187d949 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:14.501516 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298a112-ffee-4983-b5c4-96290187d949 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:14.501938 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298a112-ffee-4983-b5c4-96290187d949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:14.581771 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-6298a112-ffee-4983-b5c4-96290187d949 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:19:14.583160 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-6298a112-ffee-4983-b5c4-96290187d949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:14.708993 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:14.719057 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 688/1381] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:19:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:19:15.004955 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2fa67-67d3-4527-b199-c88121472417 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:19:15.006501 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-c2f2fa67-67d3-4527-b199-c88121472417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:15.140352 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:15.151541 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2fa67-67d3-4527-b199-c88121472417 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:19:15.274127 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eae839-0528-4f35-bc56-60ce8a1dbc78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:19:15.274854 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eae839-0528-4f35-bc56-60ce8a1dbc78 service nova] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:15.275140 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eae839-0528-4f35-bc56-60ce8a1dbc78 service nova] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:15.275412 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eae839-0528-4f35-bc56-60ce8a1dbc78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:15.277883 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-c5eae839-0528-4f35-bc56-60ce8a1dbc78 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:15.369650 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eae839-0528-4f35-bc56-60ce8a1dbc78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:19:15.371782 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eae839-0528-4f35-bc56-60ce8a1dbc78 service nova] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:19:15.381558 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:15.390989 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 694/1382] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:19:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:19:17.643190 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0a862-c5f7-4092-afa6-c930f2e08d4e None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:17.643745 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0a862-c5f7-4092-afa6-c930f2e08d4e None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:17.643903 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0a862-c5f7-4092-afa6-c930f2e08d4e None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:17.648163 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 689/1383] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:19:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:19:17.655956 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55626568-3bfe-4cb5-a5c7-e3a0d23d8cfd None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:17.656726 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55626568-3bfe-4cb5-a5c7-e3a0d23d8cfd None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:17.656947 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55626568-3bfe-4cb5-a5c7-e3a0d23d8cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:17.723490 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-55626568-3bfe-4cb5-a5c7-e3a0d23d8cfd None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:19:17.724693 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-55626568-3bfe-4cb5-a5c7-e3a0d23d8cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:17.824008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 695/1384] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:19:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:19:18.111178 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de0a862-c5f7-4092-afa6-c930f2e08d4e None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:19:18.112699 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-5de0a862-c5f7-4092-afa6-c930f2e08d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:18.263573 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de0a862-c5f7-4092-afa6-c930f2e08d4e None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:19:18.335433 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:18.381950 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab375c6e-a48c-4c7d-8b60-a5c80d0a5f69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:19:18.382559 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab375c6e-a48c-4c7d-8b60-a5c80d0a5f69 service nova] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:18.382872 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab375c6e-a48c-4c7d-8b60-a5c80d0a5f69 service nova] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:18.383150 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab375c6e-a48c-4c7d-8b60-a5c80d0a5f69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:18.384739 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-ab375c6e-a48c-4c7d-8b60-a5c80d0a5f69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:18.476075 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab375c6e-a48c-4c7d-8b60-a5c80d0a5f69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:19:18.477725 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab375c6e-a48c-4c7d-8b60-a5c80d0a5f69 service nova] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:19:18.495997 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 690/1385] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:19:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:19:20.694007 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c69359-8233-49d0-9d71-1ed5b1778834 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:20.694663 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c69359-8233-49d0-9d71-1ed5b1778834 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:20.694663 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c69359-8233-49d0-9d71-1ed5b1778834 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:20.695936 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 696/1386] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:19:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:19:20.702162 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10634d90-1582-42c7-b527-a708bb13497a None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:20.702592 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10634d90-1582-42c7-b527-a708bb13497a None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:20.702945 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10634d90-1582-42c7-b527-a708bb13497a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:20.771314 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-10634d90-1582-42c7-b527-a708bb13497a None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:19:20.772524 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-10634d90-1582-42c7-b527-a708bb13497a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:20.806529 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:20.875912 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 691/1387] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:19:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:19:21.165420 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c69359-8233-49d0-9d71-1ed5b1778834 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:19:21.167032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-53c69359-8233-49d0-9d71-1ed5b1778834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:21.243276 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:21.325174 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c69359-8233-49d0-9d71-1ed5b1778834 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:19:21.441756 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60536dbd-777b-467c-ae27-a39d434dad25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:19:21.442291 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60536dbd-777b-467c-ae27-a39d434dad25 service nova] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:21.442569 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60536dbd-777b-467c-ae27-a39d434dad25 service nova] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:21.442867 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60536dbd-777b-467c-ae27-a39d434dad25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:21.444454 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-60536dbd-777b-467c-ae27-a39d434dad25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:21.484294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:21.537162 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60536dbd-777b-467c-ae27-a39d434dad25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:19:21.538770 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60536dbd-777b-467c-ae27-a39d434dad25 service nova] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:19:21.559309 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 697/1388] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:19:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:19:23.696346 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235cb749-0033-40d6-8208-6ba8798e8059 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:23.696928 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235cb749-0033-40d6-8208-6ba8798e8059 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:23.696928 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235cb749-0033-40d6-8208-6ba8798e8059 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:23.698680 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 692/1389] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:19:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:19:23.704438 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9bb8bd-fd56-4698-a315-290147f40d73 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:23.704834 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9bb8bd-fd56-4698-a315-290147f40d73 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:23.704990 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9bb8bd-fd56-4698-a315-290147f40d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:23.772661 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0666c-519a-4a67-a448-70fe1e368366 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:23.772941 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0666c-519a-4a67-a448-70fe1e368366 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:23.773210 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-9d9bb8bd-fd56-4698-a315-290147f40d73 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:19:23.773333 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0666c-519a-4a67-a448-70fe1e368366 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:23.774428 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-9d9bb8bd-fd56-4698-a315-290147f40d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:23.774613 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 693/1390] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:19:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:19:23.780222 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1e4b9-8a2e-4eaf-afb8-a7cb05405438 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:23.780504 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1e4b9-8a2e-4eaf-afb8-a7cb05405438 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:23.780778 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1e4b9-8a2e-4eaf-afb8-a7cb05405438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:23.848539 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-8ac1e4b9-8a2e-4eaf-afb8-a7cb05405438 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:19:23.849753 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-8ac1e4b9-8a2e-4eaf-afb8-a7cb05405438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:23.874986 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 698/1391] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:19:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:19:23.948482 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 694/1392] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:19:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:19:24.184303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2cf2cb-c27e-4148-9c10-39fc1c36ff54 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:24.184678 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2cf2cb-c27e-4148-9c10-39fc1c36ff54 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:24.185015 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2cf2cb-c27e-4148-9c10-39fc1c36ff54 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:24.186495 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 699/1393] 10.222.0.52 () {54 vars in 962 bytes} [Sat Sep 24 14:19:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:19:24.192021 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17502ce8-04f7-4948-a9a7-4fd9603b248f None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:24.192380 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17502ce8-04f7-4948-a9a7-4fd9603b248f None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:24.192727 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17502ce8-04f7-4948-a9a7-4fd9603b248f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:24.209127 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:24.241544 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2cf2cb-c27e-4148-9c10-39fc1c36ff54 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:19:24.243089 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-cc2cf2cb-c27e-4148-9c10-39fc1c36ff54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:24.257118 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:24.273677 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-17502ce8-04f7-4948-a9a7-4fd9603b248f None None] MFA Rules not processed for user `e82b01dbd6e84a9d82baa2c28d77fc28`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:19:24.274904 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-17502ce8-04f7-4948-a9a7-4fd9603b248f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:24.360522 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 695/1394] 10.222.0.52 () {58 vars in 1059 bytes} [Sat Sep 24 14:19:24 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:19:24.370213 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17502ce8-04f7-4948-a9a7-4fd9603b248f None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:19:24.377581 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17502ce8-04f7-4948-a9a7-4fd9603b248f None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:19:24.397266 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2cf2cb-c27e-4148-9c10-39fc1c36ff54 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:19:24.452575 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c4a28e-7405-4ee3-85eb-4094ab83d2d8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e82b01dbd6e84a9d82baa2c28d77fc28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:19:24.453178 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4a28e-7405-4ee3-85eb-4094ab83d2d8 service glance] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:24.453451 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4a28e-7405-4ee3-85eb-4094ab83d2d8 service glance] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:24.453716 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4a28e-7405-4ee3-85eb-4094ab83d2d8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:24.455356 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-44c4a28e-7405-4ee3-85eb-4094ab83d2d8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:24.486694 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:24.491406 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:24.518387 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a3d1d-d4b4-45ee-9b34-896d2fa719e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:19:24.519023 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a3d1d-d4b4-45ee-9b34-896d2fa719e4 service nova] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:24.519299 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a3d1d-d4b4-45ee-9b34-896d2fa719e4 service nova] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:24.519564 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a3d1d-d4b4-45ee-9b34-896d2fa719e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:24.521135 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-9b9a3d1d-d4b4-45ee-9b34-896d2fa719e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:24.569307 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c4a28e-7405-4ee3-85eb-4094ab83d2d8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:19:24.570785 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c4a28e-7405-4ee3-85eb-4094ab83d2d8 service glance] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:19:24.589849 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 696/1395] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:19:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 14:19:24.613497 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9a3d1d-d4b4-45ee-9b34-896d2fa719e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:19:24.615120 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9a3d1d-d4b4-45ee-9b34-896d2fa719e4 service nova] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:19:24.633580 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 700/1396] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:19:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 14:19:25.339074 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767f161-1382-4440-b996-2c509eb85f24 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:25.339547 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767f161-1382-4440-b996-2c509eb85f24 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:25.339748 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767f161-1382-4440-b996-2c509eb85f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:25.407064 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-c767f161-1382-4440-b996-2c509eb85f24 None None] MFA Rules not processed for user `82328b7fd50649a485c6eaf27c5597c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:19:25.408281 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-c767f161-1382-4440-b996-2c509eb85f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:25.490902 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:25.496969 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 697/1397] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 14:19:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:19:25.547057 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43d85d-fe35-4e6c-a135-befe48896662 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:25.547610 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43d85d-fe35-4e6c-a135-befe48896662 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:25.547610 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43d85d-fe35-4e6c-a135-befe48896662 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:25.549073 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 701/1398] 10.222.0.52 () {54 vars in 943 bytes} [Sat Sep 24 14:19:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:19:25.555974 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e048dd-19f5-4bee-b20d-b0e91484ebf0 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:25.556261 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e048dd-19f5-4bee-b20d-b0e91484ebf0 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:25.556534 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e048dd-19f5-4bee-b20d-b0e91484ebf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:25.636328 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-91e048dd-19f5-4bee-b20d-b0e91484ebf0 None None] MFA Rules not processed for user `879f296733114f6f844a98aaefe35e92`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:19:25.637524 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-91e048dd-19f5-4bee-b20d-b0e91484ebf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:25.742410 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 698/1399] 10.222.0.52 () {58 vars in 1040 bytes} [Sat Sep 24 14:19:25 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:19:25.752675 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43d85d-fe35-4e6c-a135-befe48896662 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:19:25.754179 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-ed43d85d-fe35-4e6c-a135-befe48896662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:25.884090 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:25.928069 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43d85d-fe35-4e6c-a135-befe48896662 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:19:26.038696 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f3823-6a11-4e21-ac6f-a7a9747dbb54 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '879f296733114f6f844a98aaefe35e92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:19:26.039340 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f3823-6a11-4e21-ac6f-a7a9747dbb54 service swift] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:26.039611 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f3823-6a11-4e21-ac6f-a7a9747dbb54 service swift] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:26.039875 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f3823-6a11-4e21-ac6f-a7a9747dbb54 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:26.041966 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-c69f3823-6a11-4e21-ac6f-a7a9747dbb54 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:26.124062 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69f3823-6a11-4e21-ac6f-a7a9747dbb54 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:19:26.125649 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69f3823-6a11-4e21-ac6f-a7a9747dbb54 service swift] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:19:26.129224 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:26.136528 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 702/1400] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 14:19:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 24 14:19:26.852417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc8c6e-03bb-43c9-95a8-20b9a1492204 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:26.852941 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc8c6e-03bb-43c9-95a8-20b9a1492204 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:26.852941 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc8c6e-03bb-43c9-95a8-20b9a1492204 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:26.855978 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 699/1401] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:19:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:19:26.861514 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26c070-7cf5-46c5-8be8-f4cfc64d3fb9 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:26.861987 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26c070-7cf5-46c5-8be8-f4cfc64d3fb9 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:26.862097 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26c070-7cf5-46c5-8be8-f4cfc64d3fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:26.935156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-ab26c070-7cf5-46c5-8be8-f4cfc64d3fb9 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:19:26.936492 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-ab26c070-7cf5-46c5-8be8-f4cfc64d3fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:27.049838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 703/1402] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:19:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:19:27.336684 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecc8c6e-03bb-43c9-95a8-20b9a1492204 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:19:27.338714 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-3ecc8c6e-03bb-43c9-95a8-20b9a1492204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:27.399355 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:27.500919 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecc8c6e-03bb-43c9-95a8-20b9a1492204 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:19:27.630948 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b286c1a-6e01-4536-95dd-d1cbefb0dcbe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:19:27.631566 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b286c1a-6e01-4536-95dd-d1cbefb0dcbe service nova] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:27.631844 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b286c1a-6e01-4536-95dd-d1cbefb0dcbe service nova] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:27.632109 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b286c1a-6e01-4536-95dd-d1cbefb0dcbe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:27.634672 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-0b286c1a-6e01-4536-95dd-d1cbefb0dcbe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:27.652737 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:27.735790 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b286c1a-6e01-4536-95dd-d1cbefb0dcbe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:19:27.737390 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b286c1a-6e01-4536-95dd-d1cbefb0dcbe service nova] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:19:27.756527 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 700/1403] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:19:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:19:30.016938 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6437919-c152-4905-aaf8-4fea6a958ab8 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:30.017848 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6437919-c152-4905-aaf8-4fea6a958ab8 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:30.017848 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6437919-c152-4905-aaf8-4fea6a958ab8 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:30.022846 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 704/1404] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:19:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:19:30.024438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d856d-6a4b-4e63-9b6c-31777eadff5e None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:30.024881 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d856d-6a4b-4e63-9b6c-31777eadff5e None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:30.024974 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d856d-6a4b-4e63-9b6c-31777eadff5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:30.110473 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-704d856d-6a4b-4e63-9b6c-31777eadff5e None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:19:30.111655 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-704d856d-6a4b-4e63-9b6c-31777eadff5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:30.210228 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:30.219420 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 701/1405] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:19:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:19:30.508854 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6437919-c152-4905-aaf8-4fea6a958ab8 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:19:30.511022 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-e6437919-c152-4905-aaf8-4fea6a958ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:30.621197 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:30.695967 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6437919-c152-4905-aaf8-4fea6a958ab8 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:19:30.823849 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43129198-e22a-4988-a974-475e22799eda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:19:30.824462 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43129198-e22a-4988-a974-475e22799eda service nova] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:30.824739 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43129198-e22a-4988-a974-475e22799eda service nova] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:30.825009 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43129198-e22a-4988-a974-475e22799eda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:30.826595 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-43129198-e22a-4988-a974-475e22799eda service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:30.876932 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:30.931894 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43129198-e22a-4988-a974-475e22799eda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:19:30.933992 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43129198-e22a-4988-a974-475e22799eda service nova] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:19:30.954585 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 705/1406] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:19:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:19:33.191393 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89573dfb-560a-4b8b-90fe-931e9f7b7496 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:33.191981 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89573dfb-560a-4b8b-90fe-931e9f7b7496 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:33.191981 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89573dfb-560a-4b8b-90fe-931e9f7b7496 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:33.193278 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 702/1407] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:19:33.199234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2489b-b59a-413f-935b-56b87be338c1 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:33.199687 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2489b-b59a-413f-935b-56b87be338c1 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:33.199779 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2489b-b59a-413f-935b-56b87be338c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:33.269467 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-d4d2489b-b59a-413f-935b-56b87be338c1 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:19:33.270694 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-d4d2489b-b59a-413f-935b-56b87be338c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:33.386845 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 706/1408] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:19:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:19:33.678658 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89573dfb-560a-4b8b-90fe-931e9f7b7496 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:19:33.680576 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-89573dfb-560a-4b8b-90fe-931e9f7b7496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:33.839769 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89573dfb-560a-4b8b-90fe-931e9f7b7496 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:19:33.925476 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:33.974477 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c16cde-6e3d-423a-8f80-464823c6507d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:19:33.975373 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c16cde-6e3d-423a-8f80-464823c6507d service nova] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:33.975667 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c16cde-6e3d-423a-8f80-464823c6507d service nova] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:33.976085 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c16cde-6e3d-423a-8f80-464823c6507d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:33.977799 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-a3c16cde-6e3d-423a-8f80-464823c6507d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:34.073370 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c16cde-6e3d-423a-8f80-464823c6507d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:19:34.075312 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c16cde-6e3d-423a-8f80-464823c6507d service nova] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:19:34.094616 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 703/1409] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:19:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:19:43.381142 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=93 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:44.092300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=65 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:48.915336 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ef697-306c-4455-865c-14c325193622 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:48.915904 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ef697-306c-4455-865c-14c325193622 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:48.915904 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ef697-306c-4455-865c-14c325193622 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:48.917879 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 707/1410] 10.222.0.52 () {54 vars in 918 bytes} [Sat Sep 24 14:19:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:19:48.923112 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c21c54-f522-4249-a4b8-94f59597e0e1 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:48.923521 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c21c54-f522-4249-a4b8-94f59597e0e1 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:48.923693 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c21c54-f522-4249-a4b8-94f59597e0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:48.998565 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-91c21c54-f522-4249-a4b8-94f59597e0e1 None None] MFA Rules not processed for user `18fe384ecd454fb298dcc788f41dd44b`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:19:48.999791 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-91c21c54-f522-4249-a4b8-94f59597e0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:49.103380 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 704/1411] 10.222.0.52 () {58 vars in 1015 bytes} [Sat Sep 24 14:19:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:19:49.118008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1682fd-9172-4505-8236-a338b57e3606 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:49.118290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1682fd-9172-4505-8236-a338b57e3606 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:49.118564 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1682fd-9172-4505-8236-a338b57e3606 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:49.119946 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 708/1412] 10.222.0.52 () {54 vars in 918 bytes} [Sat Sep 24 14:19:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:19:49.124425 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70a473-4c73-4e43-b883-d7905fc385be None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:49.124711 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70a473-4c73-4e43-b883-d7905fc385be None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:49.124985 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70a473-4c73-4e43-b883-d7905fc385be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:49.197161 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:49.200208 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-3e70a473-4c73-4e43-b883-d7905fc385be None None] MFA Rules not processed for user `79eaaae97aac4282a613939e5b768376`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:19:49.201917 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-3e70a473-4c73-4e43-b883-d7905fc385be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:49.317290 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 705/1413] 10.222.0.52 () {58 vars in 1015 bytes} [Sat Sep 24 14:19:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:19:51.912106 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368864ba-36f9-4d8d-9f76-0c623bd59c58 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:51.912615 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368864ba-36f9-4d8d-9f76-0c623bd59c58 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:51.912728 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368864ba-36f9-4d8d-9f76-0c623bd59c58 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:51.913974 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 709/1414] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 14:19:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:19:51.919218 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae130c26-386f-46ed-a348-784a691382f4 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:51.919591 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae130c26-386f-46ed-a348-784a691382f4 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:51.919777 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae130c26-386f-46ed-a348-784a691382f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:51.986746 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-ae130c26-386f-46ed-a348-784a691382f4 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:19:51.987967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-ae130c26-386f-46ed-a348-784a691382f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:52.040973 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:52.082129 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 706/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 14:19:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:19:52.384548 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368864ba-36f9-4d8d-9f76-0c623bd59c58 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:19:52.386120 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-368864ba-36f9-4d8d-9f76-0c623bd59c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:52.530397 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368864ba-36f9-4d8d-9f76-0c623bd59c58 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:19:52.611709 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:52.656697 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757bcd49-3eed-4331-a79e-b109c3c12e26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:19:52.657956 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bcd49-3eed-4331-a79e-b109c3c12e26 service nova] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:52.658411 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bcd49-3eed-4331-a79e-b109c3c12e26 service nova] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:52.658694 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bcd49-3eed-4331-a79e-b109c3c12e26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:52.660343 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-757bcd49-3eed-4331-a79e-b109c3c12e26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:52.748669 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757bcd49-3eed-4331-a79e-b109c3c12e26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:19:52.750262 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757bcd49-3eed-4331-a79e-b109c3c12e26 service nova] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:19:52.767923 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 710/1416] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:19:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:19:53.912400 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce1c6b-37db-4b5b-a290-031045036b85 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:53.912960 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce1c6b-37db-4b5b-a290-031045036b85 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:53.912960 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce1c6b-37db-4b5b-a290-031045036b85 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:53.914383 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 707/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 14:19:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:19:53.921260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f9fe3-93c3-4dba-b7d1-ae36e0972d9f None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:53.921722 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f9fe3-93c3-4dba-b7d1-ae36e0972d9f None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:53.921847 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f9fe3-93c3-4dba-b7d1-ae36e0972d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:53.996863 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-ae6f9fe3-93c3-4dba-b7d1-ae36e0972d9f None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:19:53.998052 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-ae6f9fe3-93c3-4dba-b7d1-ae36e0972d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:54.024623 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:54.093494 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 711/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 14:19:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:19:54.246847 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cce1c6b-37db-4b5b-a290-031045036b85 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:19:54.248421 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-3cce1c6b-37db-4b5b-a290-031045036b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:54.392854 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cce1c6b-37db-4b5b-a290-031045036b85 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:19:54.433818 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:54.505083 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b299808e-8d70-415a-b2f7-52fc48b84173 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:19:54.505681 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299808e-8d70-415a-b2f7-52fc48b84173 service nova] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:54.505957 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299808e-8d70-415a-b2f7-52fc48b84173 service nova] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:54.506225 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299808e-8d70-415a-b2f7-52fc48b84173 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:54.507834 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-b299808e-8d70-415a-b2f7-52fc48b84173 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:54.600228 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b299808e-8d70-415a-b2f7-52fc48b84173 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:19:54.601840 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b299808e-8d70-415a-b2f7-52fc48b84173 service nova] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:19:54.619684 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 708/1419] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:19:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:19:55.749112 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d480062-5545-48f2-befa-9b82aefa39c3 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:55.749668 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d480062-5545-48f2-befa-9b82aefa39c3 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:55.749668 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d480062-5545-48f2-befa-9b82aefa39c3 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:55.751042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 712/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 14:19:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:19:55.757096 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9900d808-390e-492a-a252-26169e24dc20 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:55.757562 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9900d808-390e-492a-a252-26169e24dc20 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:55.757667 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9900d808-390e-492a-a252-26169e24dc20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:55.812689 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:55.835719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-9900d808-390e-492a-a252-26169e24dc20 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:19:55.836917 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-9900d808-390e-492a-a252-26169e24dc20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:55.932765 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 709/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 14:19:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:19:56.083550 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d480062-5545-48f2-befa-9b82aefa39c3 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:19:56.085093 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-4d480062-5545-48f2-befa-9b82aefa39c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:56.230003 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d480062-5545-48f2-befa-9b82aefa39c3 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:19:56.241693 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:56.341806 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0c31c-2cad-4568-a27e-2c999be26bdd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:19:56.342405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0c31c-2cad-4568-a27e-2c999be26bdd service nova] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:56.342683 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0c31c-2cad-4568-a27e-2c999be26bdd service nova] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:56.342984 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0c31c-2cad-4568-a27e-2c999be26bdd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:56.344554 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-63c0c31c-2cad-4568-a27e-2c999be26bdd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:56.431286 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c0c31c-2cad-4568-a27e-2c999be26bdd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:19:56.433359 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c0c31c-2cad-4568-a27e-2c999be26bdd service nova] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:19:56.451362 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 713/1422] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:19:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:19:57.546237 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326fce15-d457-4438-9449-23c5c6342540 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:57.546845 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326fce15-d457-4438-9449-23c5c6342540 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:57.546845 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326fce15-d457-4438-9449-23c5c6342540 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:57.548281 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 710/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 14:19:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:19:57.554539 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a78e4-2937-4535-9832-9fa8993b72f3 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:57.555029 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a78e4-2937-4535-9832-9fa8993b72f3 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:57.555127 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a78e4-2937-4535-9832-9fa8993b72f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:57.575204 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:57.631757 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-604a78e4-2937-4535-9832-9fa8993b72f3 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:19:57.632958 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-604a78e4-2937-4535-9832-9fa8993b72f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:57.728278 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 714/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 14:19:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:19:57.880089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326fce15-d457-4438-9449-23c5c6342540 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:19:57.881651 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-326fce15-d457-4438-9449-23c5c6342540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:57.990273 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:58.025659 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326fce15-d457-4438-9449-23c5c6342540 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:19:58.139451 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02479391-52a7-40de-ac91-0a6e375c4886 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:19:58.140061 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02479391-52a7-40de-ac91-0a6e375c4886 service nova] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:58.140337 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02479391-52a7-40de-ac91-0a6e375c4886 service nova] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:58.140608 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02479391-52a7-40de-ac91-0a6e375c4886 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:58.142203 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-02479391-52a7-40de-ac91-0a6e375c4886 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:58.217316 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:58.231743 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02479391-52a7-40de-ac91-0a6e375c4886 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:19:58.233388 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02479391-52a7-40de-ac91-0a6e375c4886 service nova] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:19:58.250703 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 711/1425] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:19:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:19:59.373040 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e289e50-7201-4435-89ed-cfbb1b6e30b6 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:59.373580 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e289e50-7201-4435-89ed-cfbb1b6e30b6 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:59.373580 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e289e50-7201-4435-89ed-cfbb1b6e30b6 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:59.374890 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 715/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 14:19:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:19:59.381140 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d611c39-626b-4ed1-b9c0-df5016f9b5c4 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:59.381545 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d611c39-626b-4ed1-b9c0-df5016f9b5c4 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:59.381697 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d611c39-626b-4ed1-b9c0-df5016f9b5c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:59.457552 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-4d611c39-626b-4ed1-b9c0-df5016f9b5c4 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:19:59.458754 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-4d611c39-626b-4ed1-b9c0-df5016f9b5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:59.554378 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 712/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 14:19:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:19:59.712896 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e289e50-7201-4435-89ed-cfbb1b6e30b6 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:19:59.714402 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-3e289e50-7201-4435-89ed-cfbb1b6e30b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:19:59.797872 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:19:59.858963 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e289e50-7201-4435-89ed-cfbb1b6e30b6 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:19:59.979004 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d62ec0-6ab4-47c4-b8a0-d07ee6253f11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:19:59.979510 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d62ec0-6ab4-47c4-b8a0-d07ee6253f11 service nova] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:19:59.979789 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d62ec0-6ab4-47c4-b8a0-d07ee6253f11 service nova] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:19:59.980056 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d62ec0-6ab4-47c4-b8a0-d07ee6253f11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:19:59.981662 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-50d62ec0-6ab4-47c4-b8a0-d07ee6253f11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:00.035095 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:20:00.075920 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d62ec0-6ab4-47c4-b8a0-d07ee6253f11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:20:00.077533 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d62ec0-6ab4-47c4-b8a0-d07ee6253f11 service nova] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:20:00.095064 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 716/1428] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:19:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:20:01.143809 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef5719-419c-4293-b5a2-85292d8ff3c1 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:01.144471 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef5719-419c-4293-b5a2-85292d8ff3c1 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:01.144471 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef5719-419c-4293-b5a2-85292d8ff3c1 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:01.145675 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 713/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 14:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:20:01.151769 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3d0c4-802a-4ec3-96ea-f8a62d623bd2 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:01.152242 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3d0c4-802a-4ec3-96ea-f8a62d623bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:01.152367 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3d0c4-802a-4ec3-96ea-f8a62d623bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:01.226856 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-74e3d0c4-802a-4ec3-96ea-f8a62d623bd2 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:20:01.228050 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-74e3d0c4-802a-4ec3-96ea-f8a62d623bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:01.325685 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 717/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 14:20:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:20:01.475716 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ef5719-419c-4293-b5a2-85292d8ff3c1 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:20:01.477148 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-91ef5719-419c-4293-b5a2-85292d8ff3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:01.524344 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:20:01.620957 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ef5719-419c-4293-b5a2-85292d8ff3c1 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:20:01.731117 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17880bb-b235-4cda-af43-cae0d74631ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:20:01.731654 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17880bb-b235-4cda-af43-cae0d74631ef service nova] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:01.731933 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17880bb-b235-4cda-af43-cae0d74631ef service nova] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:01.732205 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17880bb-b235-4cda-af43-cae0d74631ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:01.733791 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-f17880bb-b235-4cda-af43-cae0d74631ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:01.740647 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:20:01.824029 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17880bb-b235-4cda-af43-cae0d74631ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:20:01.825622 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17880bb-b235-4cda-af43-cae0d74631ef service nova] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:20:01.843453 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 714/1431] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:20:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:20:02.887096 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebab950-cb1a-4bcf-9b8e-baa549bbdf56 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:02.887599 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebab950-cb1a-4bcf-9b8e-baa549bbdf56 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:02.887702 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebab950-cb1a-4bcf-9b8e-baa549bbdf56 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:02.888980 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 718/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 14:20:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:20:02.895102 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9239cc-67ac-4fe9-8685-6831913a9c46 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:02.895541 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9239cc-67ac-4fe9-8685-6831913a9c46 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:02.895647 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9239cc-67ac-4fe9-8685-6831913a9c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:03.088099 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-fe9239cc-67ac-4fe9-8685-6831913a9c46 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:20:03.089206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-fe9239cc-67ac-4fe9-8685-6831913a9c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:03.155934 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:20:03.184257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 715/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 14:20:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:20:03.334210 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebab950-cb1a-4bcf-9b8e-baa549bbdf56 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:20:03.335842 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-8ebab950-cb1a-4bcf-9b8e-baa549bbdf56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:03.355017 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:20:03.480441 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebab950-cb1a-4bcf-9b8e-baa549bbdf56 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:20:03.565680 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:20:03.591558 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d8bb40-62a0-4ae4-9eae-0305deaf7c49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:20:03.592157 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8bb40-62a0-4ae4-9eae-0305deaf7c49 service nova] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:03.592435 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8bb40-62a0-4ae4-9eae-0305deaf7c49 service nova] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:03.592703 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8bb40-62a0-4ae4-9eae-0305deaf7c49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:03.594293 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-12d8bb40-62a0-4ae4-9eae-0305deaf7c49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:03.684234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d8bb40-62a0-4ae4-9eae-0305deaf7c49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:20:03.685830 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d8bb40-62a0-4ae4-9eae-0305deaf7c49 service nova] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:20:03.704503 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 719/1434] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:20:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:20:04.752371 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a95a2-7764-44f4-81e5-f35594505451 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:04.752937 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a95a2-7764-44f4-81e5-f35594505451 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:04.752937 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a95a2-7764-44f4-81e5-f35594505451 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:04.754236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 716/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 14:20:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:20:04.761129 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa105d-72cf-47f6-9a2d-88df42b11c55 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:04.761548 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa105d-72cf-47f6-9a2d-88df42b11c55 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:04.761658 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa105d-72cf-47f6-9a2d-88df42b11c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:04.836396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-d7fa105d-72cf-47f6-9a2d-88df42b11c55 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:20:04.838127 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-d7fa105d-72cf-47f6-9a2d-88df42b11c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:04.879727 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:20:04.934161 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 720/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 14:20:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:20:05.084365 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a95a2-7764-44f4-81e5-f35594505451 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:20:05.085797 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-0d5a95a2-7764-44f4-81e5-f35594505451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:05.229126 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a95a2-7764-44f4-81e5-f35594505451 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:20:05.282262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:20:05.340257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba9a04-bb06-4e05-b098-0aa1c05a28bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:20:05.340788 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba9a04-bb06-4e05-b098-0aa1c05a28bf service nova] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:05.341063 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba9a04-bb06-4e05-b098-0aa1c05a28bf service nova] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:05.341328 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba9a04-bb06-4e05-b098-0aa1c05a28bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:05.342937 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-12ba9a04-bb06-4e05-b098-0aa1c05a28bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:05.432244 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ba9a04-bb06-4e05-b098-0aa1c05a28bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:20:05.433806 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ba9a04-bb06-4e05-b098-0aa1c05a28bf service nova] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:20:05.451032 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 717/1437] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:20:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:20:06.505277 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79a8f3-98a8-4235-8278-4472ccc0b69b None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:06.505848 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79a8f3-98a8-4235-8278-4472ccc0b69b None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:06.505848 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79a8f3-98a8-4235-8278-4472ccc0b69b None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:06.507242 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 721/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 14:20:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:20:06.513749 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45942d-076b-4be9-bfeb-f49f6fc2dcd6 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:06.514182 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45942d-076b-4be9-bfeb-f49f6fc2dcd6 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:06.514286 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45942d-076b-4be9-bfeb-f49f6fc2dcd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:06.588548 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-1e45942d-076b-4be9-bfeb-f49f6fc2dcd6 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:20:06.589666 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-1e45942d-076b-4be9-bfeb-f49f6fc2dcd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:06.591667 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:20:06.689801 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 718/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 14:20:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:20:06.840654 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79a8f3-98a8-4235-8278-4472ccc0b69b None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:20:06.842148 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-4c79a8f3-98a8-4235-8278-4472ccc0b69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:06.984770 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79a8f3-98a8-4235-8278-4472ccc0b69b None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:20:07.011838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:20:07.097947 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1495f29f-453f-47dc-8972-da9531a2210f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:20:07.098466 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495f29f-453f-47dc-8972-da9531a2210f service nova] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:07.098738 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495f29f-453f-47dc-8972-da9531a2210f service nova] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:07.099052 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495f29f-453f-47dc-8972-da9531a2210f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:07.100622 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-1495f29f-453f-47dc-8972-da9531a2210f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:07.188320 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1495f29f-453f-47dc-8972-da9531a2210f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:20:07.189934 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1495f29f-453f-47dc-8972-da9531a2210f service nova] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:20:07.207502 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 722/1440] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:20:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:20:09.270202 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa389d-23b5-4207-bb26-257cfcadd9cb None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:09.270743 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa389d-23b5-4207-bb26-257cfcadd9cb None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:09.270743 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa389d-23b5-4207-bb26-257cfcadd9cb None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:09.272165 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 719/1441] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 14:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:20:09.278801 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b5599-44d0-4e4a-a6e6-7f01025c64a9 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:09.279342 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b5599-44d0-4e4a-a6e6-7f01025c64a9 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:09.279843 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b5599-44d0-4e4a-a6e6-7f01025c64a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:09.322436 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:20:09.348024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-d30b5599-44d0-4e4a-a6e6-7f01025c64a9 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:20:09.349234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-d30b5599-44d0-4e4a-a6e6-7f01025c64a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:09.446544 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 723/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 14:20:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:20:09.750542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aa389d-23b5-4207-bb26-257cfcadd9cb None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:20:09.752120 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-d3aa389d-23b5-4207-bb26-257cfcadd9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:09.855232 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aa389d-23b5-4207-bb26-257cfcadd9cb None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:20:09.887282 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:20:09.928759 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440c34f7-a865-4f9a-bfb1-a8636756313d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:20:09.929362 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c34f7-a865-4f9a-bfb1-a8636756313d service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:09.929642 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c34f7-a865-4f9a-bfb1-a8636756313d service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:09.929911 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c34f7-a865-4f9a-bfb1-a8636756313d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:09.932061 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-440c34f7-a865-4f9a-bfb1-a8636756313d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:10.050466 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440c34f7-a865-4f9a-bfb1-a8636756313d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:20:10.051930 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440c34f7-a865-4f9a-bfb1-a8636756313d service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:20:10.069737 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 720/1443] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:20:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:20:12.387940 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2992e-042c-437e-9043-1c3e9d4d3575 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:12.388439 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2992e-042c-437e-9043-1c3e9d4d3575 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:12.388553 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2992e-042c-437e-9043-1c3e9d4d3575 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:12.389812 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 724/1444] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 14:20:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:20:12.395192 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c40b4-4707-446c-980d-5997e94026ed None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:12.395586 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c40b4-4707-446c-980d-5997e94026ed None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:12.395751 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c40b4-4707-446c-980d-5997e94026ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:12.461142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-c45c40b4-4707-446c-980d-5997e94026ed None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:20:12.462340 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-c45c40b4-4707-446c-980d-5997e94026ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:12.475550 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:20:12.560673 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 721/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 14:20:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:20:12.868402 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2992e-042c-437e-9043-1c3e9d4d3575 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:20:12.870617 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-cbd2992e-042c-437e-9043-1c3e9d4d3575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:12.974529 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2992e-042c-437e-9043-1c3e9d4d3575 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:20:12.987529 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:20:13.049290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b0d2a-faa9-4d95-b46f-e20a9f29a9e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:20:13.049897 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b0d2a-faa9-4d95-b46f-e20a9f29a9e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:13.050174 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b0d2a-faa9-4d95-b46f-e20a9f29a9e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:13.050496 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b0d2a-faa9-4d95-b46f-e20a9f29a9e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:13.052122 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-046b0d2a-faa9-4d95-b46f-e20a9f29a9e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:13.161663 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046b0d2a-faa9-4d95-b46f-e20a9f29a9e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:20:13.163138 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046b0d2a-faa9-4d95-b46f-e20a9f29a9e9 service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:20:13.181122 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 725/1446] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:20:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:20:15.671457 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b31793-09d9-4c96-9d69-f24811694255 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:15.672029 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b31793-09d9-4c96-9d69-f24811694255 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:15.672029 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b31793-09d9-4c96-9d69-f24811694255 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:15.673344 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 722/1447] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 14:20:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:20:15.678708 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c522c3-8af7-4626-937a-56f5f46ad704 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:15.679282 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c522c3-8af7-4626-937a-56f5f46ad704 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:15.679282 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c522c3-8af7-4626-937a-56f5f46ad704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:15.727313 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:20:15.745435 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-e4c522c3-8af7-4626-937a-56f5f46ad704 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:20:15.746623 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-e4c522c3-8af7-4626-937a-56f5f46ad704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:15.845658 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 726/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 14:20:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:20:16.137183 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b31793-09d9-4c96-9d69-f24811694255 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:20:16.138695 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-e1b31793-09d9-4c96-9d69-f24811694255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:16.246545 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b31793-09d9-4c96-9d69-f24811694255 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:20:16.286764 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:20:16.319755 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e90993c-056c-4fa2-ba01-f3f90a4d978c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:20:16.320354 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90993c-056c-4fa2-ba01-f3f90a4d978c service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:16.320628 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90993c-056c-4fa2-ba01-f3f90a4d978c service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:16.320894 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90993c-056c-4fa2-ba01-f3f90a4d978c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:16.323029 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-9e90993c-056c-4fa2-ba01-f3f90a4d978c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:16.430418 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e90993c-056c-4fa2-ba01-f3f90a4d978c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:20:16.431829 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e90993c-056c-4fa2-ba01-f3f90a4d978c service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:20:16.450551 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 723/1449] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:20:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:20:18.704326 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a6cbb-2f5c-4d21-a7d1-699a0cb944b0 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:18.704918 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a6cbb-2f5c-4d21-a7d1-699a0cb944b0 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:18.704918 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a6cbb-2f5c-4d21-a7d1-699a0cb944b0 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:18.706194 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 727/1450] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 14:20:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:20:18.711609 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e342575-2ed5-4eba-8858-9584747e1ce9 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:18.712056 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e342575-2ed5-4eba-8858-9584747e1ce9 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:18.712162 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e342575-2ed5-4eba-8858-9584747e1ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:18.778902 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-1e342575-2ed5-4eba-8858-9584747e1ce9 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:20:18.780115 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-1e342575-2ed5-4eba-8858-9584747e1ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:18.801095 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:20:18.876898 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 724/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 14:20:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:20:19.159657 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a6cbb-2f5c-4d21-a7d1-699a0cb944b0 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:20:19.161206 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-513a6cbb-2f5c-4d21-a7d1-699a0cb944b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:19.262604 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a6cbb-2f5c-4d21-a7d1-699a0cb944b0 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:20:19.282209 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:20:19.335142 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb0692b-378a-436b-9b6e-d3195fb88568 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:20:19.335740 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0692b-378a-436b-9b6e-d3195fb88568 service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:19.336031 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0692b-378a-436b-9b6e-d3195fb88568 service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:19.336282 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0692b-378a-436b-9b6e-d3195fb88568 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:19.338485 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-9cb0692b-378a-436b-9b6e-d3195fb88568 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:19.444686 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb0692b-378a-436b-9b6e-d3195fb88568 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:20:19.446551 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb0692b-378a-436b-9b6e-d3195fb88568 service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:20:19.465440 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 728/1452] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:20:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:20:21.660313 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a540825-dad7-41f2-b361-a2947617e2c5 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:21.660885 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a540825-dad7-41f2-b361-a2947617e2c5 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:21.660885 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a540825-dad7-41f2-b361-a2947617e2c5 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:21.662182 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 725/1453] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 14:20:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:20:21.667554 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c126e06-0d63-434c-854b-a48e8074eab1 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:21.667938 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c126e06-0d63-434c-854b-a48e8074eab1 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:21.668097 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c126e06-0d63-434c-854b-a48e8074eab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:21.731067 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:20:21.733830 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-8c126e06-0d63-434c-854b-a48e8074eab1 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:20:21.735523 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-8c126e06-0d63-434c-854b-a48e8074eab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:21.830914 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 729/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 14:20:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:20:22.110648 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a540825-dad7-41f2-b361-a2947617e2c5 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:20:22.112203 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-6a540825-dad7-41f2-b361-a2947617e2c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:22.214406 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a540825-dad7-41f2-b361-a2947617e2c5 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:20:22.263210 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:20:22.288817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0706a900-fdb8-405e-a9a5-ee4b73097f08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:20:22.289424 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706a900-fdb8-405e-a9a5-ee4b73097f08 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:22.289701 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706a900-fdb8-405e-a9a5-ee4b73097f08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:22.289970 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706a900-fdb8-405e-a9a5-ee4b73097f08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:22.292111 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-0706a900-fdb8-405e-a9a5-ee4b73097f08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:22.399335 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0706a900-fdb8-405e-a9a5-ee4b73097f08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:20:22.400791 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0706a900-fdb8-405e-a9a5-ee4b73097f08 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:20:22.419271 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 726/1455] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:20:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:20:24.623305 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1c7ac-7fc7-4588-9330-aa4cbcd81c40 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:24.623849 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1c7ac-7fc7-4588-9330-aa4cbcd81c40 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:24.623849 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1c7ac-7fc7-4588-9330-aa4cbcd81c40 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:24.625124 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 730/1456] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 14:20:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:20:24.630637 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaaf31c-7cea-4b08-8e13-7f9e1d3e2d57 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:24.631090 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaaf31c-7cea-4b08-8e13-7f9e1d3e2d57 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:24.631207 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaaf31c-7cea-4b08-8e13-7f9e1d3e2d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:24.695006 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-ecaaf31c-7cea-4b08-8e13-7f9e1d3e2d57 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:20:24.696192 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-ecaaf31c-7cea-4b08-8e13-7f9e1d3e2d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:24.725191 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:20:24.791291 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 727/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 14:20:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:20:25.083423 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e1c7ac-7fc7-4588-9330-aa4cbcd81c40 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:20:25.084986 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-c0e1c7ac-7fc7-4588-9330-aa4cbcd81c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:25.188868 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e1c7ac-7fc7-4588-9330-aa4cbcd81c40 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:20:25.216987 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:20:25.262293 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a3ea9-6aa5-4205-ae52-3ca05f8e59bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:20:25.262930 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a3ea9-6aa5-4205-ae52-3ca05f8e59bc service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:25.263208 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a3ea9-6aa5-4205-ae52-3ca05f8e59bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:25.263476 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a3ea9-6aa5-4205-ae52-3ca05f8e59bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:25.265050 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-a16a3ea9-6aa5-4205-ae52-3ca05f8e59bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:25.371805 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16a3ea9-6aa5-4205-ae52-3ca05f8e59bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:20:25.373161 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16a3ea9-6aa5-4205-ae52-3ca05f8e59bc service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:20:25.390629 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 731/1458] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:20:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:20:27.586041 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10473a9-3b30-4a0d-95a6-e5fa7e8ace70 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:27.586590 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10473a9-3b30-4a0d-95a6-e5fa7e8ace70 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:27.586590 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10473a9-3b30-4a0d-95a6-e5fa7e8ace70 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:27.588094 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 728/1459] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 14:20:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:20:27.593268 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a282b-517a-4553-8681-16dee90f2291 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:27.593672 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a282b-517a-4553-8681-16dee90f2291 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:27.593834 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a282b-517a-4553-8681-16dee90f2291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:27.659570 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-545a282b-517a-4553-8681-16dee90f2291 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:20:27.660774 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-545a282b-517a-4553-8681-16dee90f2291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:27.669551 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:20:27.756131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 732/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 14:20:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:20:28.037485 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10473a9-3b30-4a0d-95a6-e5fa7e8ace70 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:20:28.039242 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-c10473a9-3b30-4a0d-95a6-e5fa7e8ace70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:28.143794 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10473a9-3b30-4a0d-95a6-e5fa7e8ace70 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:20:28.198682 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:20:28.215990 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2093321d-bb92-4cf9-afa0-dff7c7b4de88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:20:28.216591 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2093321d-bb92-4cf9-afa0-dff7c7b4de88 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:28.216864 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2093321d-bb92-4cf9-afa0-dff7c7b4de88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:28.217130 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2093321d-bb92-4cf9-afa0-dff7c7b4de88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:28.218695 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-2093321d-bb92-4cf9-afa0-dff7c7b4de88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:28.329108 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2093321d-bb92-4cf9-afa0-dff7c7b4de88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:20:28.330465 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2093321d-bb92-4cf9-afa0-dff7c7b4de88 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:20:28.348545 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 729/1461] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:20:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:20:30.550160 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a75f1a-4c10-418c-b63a-9d8acaaa8c93 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:30.550694 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a75f1a-4c10-418c-b63a-9d8acaaa8c93 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:30.550968 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a75f1a-4c10-418c-b63a-9d8acaaa8c93 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:30.552336 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 733/1462] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 14:20:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:20:30.557708 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935636a-5b86-4675-bc35-e01d6c7ed6a8 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:30.558117 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935636a-5b86-4675-bc35-e01d6c7ed6a8 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:30.558257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935636a-5b86-4675-bc35-e01d6c7ed6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:30.625442 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-c935636a-5b86-4675-bc35-e01d6c7ed6a8 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:20:30.626651 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-c935636a-5b86-4675-bc35-e01d6c7ed6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:30.666213 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:20:30.725687 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 730/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 14:20:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:20:31.004703 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a75f1a-4c10-418c-b63a-9d8acaaa8c93 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:20:31.006301 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-83a75f1a-4c10-418c-b63a-9d8acaaa8c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:31.109607 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a75f1a-4c10-418c-b63a-9d8acaaa8c93 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:20:31.146921 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:20:31.182710 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aab754-b209-429a-9af1-958c77ec4986 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:20:31.183337 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aab754-b209-429a-9af1-958c77ec4986 service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:31.183614 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aab754-b209-429a-9af1-958c77ec4986 service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:31.183883 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aab754-b209-429a-9af1-958c77ec4986 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:31.185468 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-03aab754-b209-429a-9af1-958c77ec4986 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:31.292381 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aab754-b209-429a-9af1-958c77ec4986 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:20:31.293756 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aab754-b209-429a-9af1-958c77ec4986 service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:20:31.315366 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 734/1464] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:20:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:20:33.508850 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ed0c9-2965-4379-8549-a9084dd2b09f None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:33.510011 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ed0c9-2965-4379-8549-a9084dd2b09f None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:33.510011 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ed0c9-2965-4379-8549-a9084dd2b09f None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:33.511284 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 731/1465] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 14:20:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:20:33.516694 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85404f-ee80-43fe-9e12-8de112747fbc None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:33.517153 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85404f-ee80-43fe-9e12-8de112747fbc None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:33.517260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85404f-ee80-43fe-9e12-8de112747fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:33.583521 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-5b85404f-ee80-43fe-9e12-8de112747fbc None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:20:33.585590 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-5b85404f-ee80-43fe-9e12-8de112747fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:33.604048 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:20:33.696865 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 735/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 14:20:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:20:33.986554 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8ed0c9-2965-4379-8549-a9084dd2b09f None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:20:33.988109 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-7e8ed0c9-2965-4379-8549-a9084dd2b09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:34.088980 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8ed0c9-2965-4379-8549-a9084dd2b09f None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:20:34.154540 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:20:34.161843 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d440af2f-9de2-4c87-9920-cdd8604206b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:20:34.162446 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440af2f-9de2-4c87-9920-cdd8604206b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:34.162731 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440af2f-9de2-4c87-9920-cdd8604206b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:34.163031 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440af2f-9de2-4c87-9920-cdd8604206b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:34.164626 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-d440af2f-9de2-4c87-9920-cdd8604206b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:34.272184 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d440af2f-9de2-4c87-9920-cdd8604206b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:20:34.273609 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d440af2f-9de2-4c87-9920-cdd8604206b4 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:20:34.292337 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 732/1467] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:20:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:20:36.452717 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9fb857-52c5-45a8-9040-768ecbbfeacb None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:36.453269 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9fb857-52c5-45a8-9040-768ecbbfeacb None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:36.453269 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9fb857-52c5-45a8-9040-768ecbbfeacb None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:36.454621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 736/1468] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 14:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:20:36.459833 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40759fed-fbca-48dd-b728-8557b54b5dd4 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:36.460252 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40759fed-fbca-48dd-b728-8557b54b5dd4 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:36.460401 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40759fed-fbca-48dd-b728-8557b54b5dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:36.527548 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-40759fed-fbca-48dd-b728-8557b54b5dd4 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:20:36.528740 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-40759fed-fbca-48dd-b728-8557b54b5dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:36.576436 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:20:36.624343 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 733/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 14:20:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:20:36.905866 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9fb857-52c5-45a8-9040-768ecbbfeacb None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:20:36.907446 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-db9fb857-52c5-45a8-9040-768ecbbfeacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:37.009593 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9fb857-52c5-45a8-9040-768ecbbfeacb None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:20:37.058504 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:20:37.084296 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9633c42a-796f-415c-affb-50ac80d100e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:20:37.084890 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633c42a-796f-415c-affb-50ac80d100e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:37.085162 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633c42a-796f-415c-affb-50ac80d100e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:37.085424 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633c42a-796f-415c-affb-50ac80d100e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:37.087532 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-9633c42a-796f-415c-affb-50ac80d100e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:37.192983 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9633c42a-796f-415c-affb-50ac80d100e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:20:37.194336 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9633c42a-796f-415c-affb-50ac80d100e1 service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:20:37.212641 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 737/1470] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:20:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:20:46.622840 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=22 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:20:47.209939 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=66 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:20:57.134405 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f36806-11f5-47b4-81ba-2301cc3b4848 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:57.135032 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f36806-11f5-47b4-81ba-2301cc3b4848 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:57.135032 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f36806-11f5-47b4-81ba-2301cc3b4848 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:57.136335 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 734/1471] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:20:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:20:57.141675 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51122c-7c0d-4878-b9ab-7cadb98bd63c None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:57.142095 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51122c-7c0d-4878-b9ab-7cadb98bd63c None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:57.142194 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51122c-7c0d-4878-b9ab-7cadb98bd63c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:57.207147 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-df51122c-7c0d-4878-b9ab-7cadb98bd63c None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:20:57.208339 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-df51122c-7c0d-4878-b9ab-7cadb98bd63c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:57.304196 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 738/1472] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:20:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:20:57.612350 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d78a57-5862-473c-8566-552357580a16 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:57.612632 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d78a57-5862-473c-8566-552357580a16 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:57.612931 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d78a57-5862-473c-8566-552357580a16 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:57.614291 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 735/1473] 10.222.0.52 () {54 vars in 962 bytes} [Sat Sep 24 14:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:20:57.620029 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c339e88-b174-4c5b-b857-33ceb79c3331 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:57.620311 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c339e88-b174-4c5b-b857-33ceb79c3331 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:57.620577 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c339e88-b174-4c5b-b857-33ceb79c3331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:57.694863 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-2c339e88-b174-4c5b-b857-33ceb79c3331 None None] MFA Rules not processed for user `e82b01dbd6e84a9d82baa2c28d77fc28`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:20:57.696055 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-2c339e88-b174-4c5b-b857-33ceb79c3331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:57.716514 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:20:57.770690 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 739/1474] 10.222.0.52 () {58 vars in 1059 bytes} [Sat Sep 24 14:20:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:20:57.779684 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d78a57-5862-473c-8566-552357580a16 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:20:57.781229 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-37d78a57-5862-473c-8566-552357580a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:57.883596 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d78a57-5862-473c-8566-552357580a16 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:20:57.955116 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51be085-8526-43c3-8a6a-8f9fe9a1082b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e82b01dbd6e84a9d82baa2c28d77fc28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:20:57.955716 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51be085-8526-43c3-8a6a-8f9fe9a1082b service glance] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:20:57.956114 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51be085-8526-43c3-8a6a-8f9fe9a1082b service glance] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:20:57.956387 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51be085-8526-43c3-8a6a-8f9fe9a1082b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:20:57.957959 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-a51be085-8526-43c3-8a6a-8f9fe9a1082b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:20:58.010246 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:20:58.066372 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51be085-8526-43c3-8a6a-8f9fe9a1082b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:20:58.067875 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51be085-8526-43c3-8a6a-8f9fe9a1082b service glance] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:20:58.085859 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 736/1475] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:20:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:21:00.311869 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c2580f-4ecb-4b92-9b34-dc5aef5aef23 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:21:00.312432 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c2580f-4ecb-4b92-9b34-dc5aef5aef23 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:21:00.312432 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c2580f-4ecb-4b92-9b34-dc5aef5aef23 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:21:00.313758 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 740/1476] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:21:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:21:00.318970 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2092279-a889-400f-9ae0-9a801c750c93 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:21:00.319390 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2092279-a889-400f-9ae0-9a801c750c93 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:21:00.319524 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2092279-a889-400f-9ae0-9a801c750c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:21:00.385125 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-a2092279-a889-400f-9ae0-9a801c750c93 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:21:00.386423 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-a2092279-a889-400f-9ae0-9a801c750c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:21:00.479612 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 737/1477] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:21:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:21:00.759604 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c2580f-4ecb-4b92-9b34-dc5aef5aef23 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:21:00.761143 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-e5c2580f-4ecb-4b92-9b34-dc5aef5aef23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:21:00.891720 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c2580f-4ecb-4b92-9b34-dc5aef5aef23 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:21:00.942862 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:21:00.987668 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78670fa-cd8b-4f55-8ec7-d44c357a7a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:21:00.988380 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78670fa-cd8b-4f55-8ec7-d44c357a7a6c admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:21:00.988663 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78670fa-cd8b-4f55-8ec7-d44c357a7a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:21:00.988938 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78670fa-cd8b-4f55-8ec7-d44c357a7a6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:21:00.989570 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78670fa-cd8b-4f55-8ec7-d44c357a7a6c admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:21:00.991403 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: WARNING py.warnings [None req-d78670fa-cd8b-4f55-8ec7-d44c357a7a6c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 14:21:00.991403 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: warnings.warn(msg) Sep 24 14:21:00.991403 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: Sep 24 14:21:00.992544 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78670fa-cd8b-4f55-8ec7-d44c357a7a6c admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:21:01.011200 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 741/1478] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 14:21:00 2022] GET /identity/v3/projects => generated 2553 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:21:03.155402 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506d220-50df-43e5-986b-c73ecb3ef8b4 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:21:03.155999 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506d220-50df-43e5-986b-c73ecb3ef8b4 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:21:03.155999 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506d220-50df-43e5-986b-c73ecb3ef8b4 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:21:03.157291 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 738/1479] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:21:03.162707 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4bb30a-d2c4-4750-a546-97d9fbd8b500 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:21:03.163187 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4bb30a-d2c4-4750-a546-97d9fbd8b500 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:21:03.163288 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4bb30a-d2c4-4750-a546-97d9fbd8b500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:21:03.229144 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-4b4bb30a-d2c4-4750-a546-97d9fbd8b500 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:21:03.230324 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-4b4bb30a-d2c4-4750-a546-97d9fbd8b500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:21:03.324351 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 742/1480] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:21:03.615955 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a506d220-50df-43e5-986b-c73ecb3ef8b4 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:21:03.617629 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-a506d220-50df-43e5-986b-c73ecb3ef8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:21:03.627562 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:21:03.760467 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a506d220-50df-43e5-986b-c73ecb3ef8b4 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:21:03.845536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:21:03.877359 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87340780-c39e-4855-9d44-7350947f79e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:21:03.877892 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87340780-c39e-4855-9d44-7350947f79e4 service nova] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:21:03.878166 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87340780-c39e-4855-9d44-7350947f79e4 service nova] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:21:03.878434 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87340780-c39e-4855-9d44-7350947f79e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:21:03.880042 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-87340780-c39e-4855-9d44-7350947f79e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:21:03.971626 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87340780-c39e-4855-9d44-7350947f79e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:21:03.973219 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87340780-c39e-4855-9d44-7350947f79e4 service nova] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:21:03.991062 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 739/1481] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:21:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:21:06.308453 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d8dac-50f8-4b5b-af75-a414e4da84ca None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:21:06.309039 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d8dac-50f8-4b5b-af75-a414e4da84ca None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:21:06.309039 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d8dac-50f8-4b5b-af75-a414e4da84ca None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:21:06.310328 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 743/1482] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:21:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:21:06.315540 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140e778-9b5c-4337-b90d-66bc46d56fe9 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:21:06.316028 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140e778-9b5c-4337-b90d-66bc46d56fe9 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:21:06.316136 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140e778-9b5c-4337-b90d-66bc46d56fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:21:06.380468 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-d140e778-9b5c-4337-b90d-66bc46d56fe9 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:21:06.381657 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-d140e778-9b5c-4337-b90d-66bc46d56fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:21:06.427620 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:21:06.476102 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 740/1483] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:21:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:21:06.765700 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d8dac-50f8-4b5b-af75-a414e4da84ca None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:21:06.767300 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-011d8dac-50f8-4b5b-af75-a414e4da84ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:21:06.791144 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:21:06.870680 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d8dac-50f8-4b5b-af75-a414e4da84ca None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:21:06.944517 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939f431-807d-4cf3-8122-8783bfd85713 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:21:06.945115 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939f431-807d-4cf3-8122-8783bfd85713 service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:21:06.945397 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939f431-807d-4cf3-8122-8783bfd85713 service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:21:06.946035 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939f431-807d-4cf3-8122-8783bfd85713 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:21:06.948156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-d939f431-807d-4cf3-8122-8783bfd85713 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:21:07.025092 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:21:07.060139 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d939f431-807d-4cf3-8122-8783bfd85713 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:21:07.061581 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d939f431-807d-4cf3-8122-8783bfd85713 service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:21:07.079891 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 744/1484] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:21:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:21:09.280355 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a0188-5828-41ff-8b65-48c7165b05d5 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:21:09.280926 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a0188-5828-41ff-8b65-48c7165b05d5 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:21:09.280926 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a0188-5828-41ff-8b65-48c7165b05d5 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:21:09.282243 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 741/1485] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:21:09.287693 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d8478-4d66-45c7-80b2-d0a112d97efc None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:21:09.288141 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d8478-4d66-45c7-80b2-d0a112d97efc None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:21:09.288245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d8478-4d66-45c7-80b2-d0a112d97efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:21:09.353789 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-437d8478-4d66-45c7-80b2-d0a112d97efc None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:21:09.355015 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-437d8478-4d66-45c7-80b2-d0a112d97efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:21:09.448916 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 745/1486] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:21:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:21:09.741398 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a0188-5828-41ff-8b65-48c7165b05d5 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:21:09.743608 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-3e2a0188-5828-41ff-8b65-48c7165b05d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:21:09.845084 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a0188-5828-41ff-8b65-48c7165b05d5 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:21:09.917536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:21:09.919074 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af25d6-1f2e-415c-a714-34b07ac128f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:21:09.919855 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af25d6-1f2e-415c-a714-34b07ac128f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:21:09.920382 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af25d6-1f2e-415c-a714-34b07ac128f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:21:09.920841 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af25d6-1f2e-415c-a714-34b07ac128f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:21:09.922594 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-34af25d6-1f2e-415c-a714-34b07ac128f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:21:10.029764 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af25d6-1f2e-415c-a714-34b07ac128f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:21:10.031158 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af25d6-1f2e-415c-a714-34b07ac128f9 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:21:10.049459 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 742/1487] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:21:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:21:12.295715 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c22aef1-c503-4fbe-b50b-d9379db13a67 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:21:12.296310 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c22aef1-c503-4fbe-b50b-d9379db13a67 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:21:12.296527 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c22aef1-c503-4fbe-b50b-d9379db13a67 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:21:12.297891 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 746/1488] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:21:12.303269 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e3454-ef42-4adf-ab96-a97c53649d73 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:21:12.303715 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e3454-ef42-4adf-ab96-a97c53649d73 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:21:12.303825 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e3454-ef42-4adf-ab96-a97c53649d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:21:12.369430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-793e3454-ef42-4adf-ab96-a97c53649d73 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:21:12.370631 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-793e3454-ef42-4adf-ab96-a97c53649d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:21:12.423252 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:21:12.468169 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 743/1489] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:21:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:21:12.761529 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c22aef1-c503-4fbe-b50b-d9379db13a67 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:21:12.763087 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-0c22aef1-c503-4fbe-b50b-d9379db13a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:21:12.800857 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:21:12.873716 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c22aef1-c503-4fbe-b50b-d9379db13a67 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:21:12.950101 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84c23b-a683-45d8-9096-3804cd4331c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:21:12.950703 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84c23b-a683-45d8-9096-3804cd4331c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:21:12.951008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84c23b-a683-45d8-9096-3804cd4331c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:21:12.951281 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84c23b-a683-45d8-9096-3804cd4331c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:21:12.952875 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-3b84c23b-a683-45d8-9096-3804cd4331c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:21:13.034748 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:21:13.064606 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b84c23b-a683-45d8-9096-3804cd4331c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:21:13.066054 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b84c23b-a683-45d8-9096-3804cd4331c5 service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:21:13.084177 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 747/1490] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:21:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:21:15.451049 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8adcfd-4edf-46cd-8f87-516bc53611b6 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:21:15.451614 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8adcfd-4edf-46cd-8f87-516bc53611b6 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:21:15.451614 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8adcfd-4edf-46cd-8f87-516bc53611b6 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:21:15.452976 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 744/1491] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 14:21:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:21:15.458405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04259236-5d79-429a-8846-a0f3c92b2fe9 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:21:15.458801 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04259236-5d79-429a-8846-a0f3c92b2fe9 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:21:15.458978 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04259236-5d79-429a-8846-a0f3c92b2fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:21:15.524506 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-04259236-5d79-429a-8846-a0f3c92b2fe9 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:21:15.525693 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-04259236-5d79-429a-8846-a0f3c92b2fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:21:15.621139 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 748/1492] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 14:21:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:21:15.912978 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8adcfd-4edf-46cd-8f87-516bc53611b6 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:21:15.914532 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-cc8adcfd-4edf-46cd-8f87-516bc53611b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:21:16.017627 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8adcfd-4edf-46cd-8f87-516bc53611b6 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:21:16.090902 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9329042f-a7fd-4350-bb4c-77d030efb5d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:21:16.091436 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9329042f-a7fd-4350-bb4c-77d030efb5d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:21:16.091714 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9329042f-a7fd-4350-bb4c-77d030efb5d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:21:16.091979 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9329042f-a7fd-4350-bb4c-77d030efb5d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:21:16.093563 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-9329042f-a7fd-4350-bb4c-77d030efb5d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:21:16.101999 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:21:16.201784 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9329042f-a7fd-4350-bb4c-77d030efb5d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:21:16.203272 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9329042f-a7fd-4350-bb4c-77d030efb5d7 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:21:16.221972 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 745/1493] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:21:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:21:25.619616 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=82 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:21:26.219627 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=50 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:22:31.736175 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710d2b7-dd42-442f-8d46-4a6fdbba0132 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:22:31.736755 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710d2b7-dd42-442f-8d46-4a6fdbba0132 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:22:31.736755 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710d2b7-dd42-442f-8d46-4a6fdbba0132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:22:31.811354 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-7710d2b7-dd42-442f-8d46-4a6fdbba0132 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:22:31.812535 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-7710d2b7-dd42-442f-8d46-4a6fdbba0132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:22:31.908234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 749/1494] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:22:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:22:31.916944 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925f9d2-0027-4247-8aa6-307092716a37 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:22:31.917436 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925f9d2-0027-4247-8aa6-307092716a37 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:22:31.917543 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925f9d2-0027-4247-8aa6-307092716a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:22:31.993547 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-c925f9d2-0027-4247-8aa6-307092716a37 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:22:31.994662 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-c925f9d2-0027-4247-8aa6-307092716a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:22:32.089012 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 746/1495] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:22:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:22:32.093507 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710d2b7-dd42-442f-8d46-4a6fdbba0132 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:22:32.094701 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-7710d2b7-dd42-442f-8d46-4a6fdbba0132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:22:32.106474 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710d2b7-dd42-442f-8d46-4a6fdbba0132 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:22:32.284030 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:22:32.318067 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33df32-cda2-4315-bdb1-cba3a4c3c357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:22:32.318674 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33df32-cda2-4315-bdb1-cba3a4c3c357 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:22:32.318995 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33df32-cda2-4315-bdb1-cba3a4c3c357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:22:32.319268 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33df32-cda2-4315-bdb1-cba3a4c3c357 admin admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:22:32.319907 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a33df32-cda2-4315-bdb1-cba3a4c3c357 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:22:32.320214 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a33df32-cda2-4315-bdb1-cba3a4c3c357 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:22:32.321738 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: WARNING py.warnings [None req-3a33df32-cda2-4315-bdb1-cba3a4c3c357 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 14:22:32.321738 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: warnings.warn(msg) Sep 24 14:22:32.321738 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: Sep 24 14:22:32.322617 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a33df32-cda2-4315-bdb1-cba3a4c3c357 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:22:32.333549 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 750/1496] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:22:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:22:32.342466 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925f9d2-0027-4247-8aa6-307092716a37 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:22:32.351417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925f9d2-0027-4247-8aa6-307092716a37 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:22:32.412046 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:22:32.445154 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4271e011-2ee7-449b-bff5-5932144a97c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:22:32.446303 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271e011-2ee7-449b-bff5-5932144a97c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:22:32.446578 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271e011-2ee7-449b-bff5-5932144a97c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:22:32.446871 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271e011-2ee7-449b-bff5-5932144a97c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:22:32.447492 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4271e011-2ee7-449b-bff5-5932144a97c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:22:32.449216 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: WARNING py.warnings [None req-4271e011-2ee7-449b-bff5-5932144a97c2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 14:22:32.449216 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: warnings.warn(msg) Sep 24 14:22:32.449216 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: Sep 24 14:22:32.450393 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4271e011-2ee7-449b-bff5-5932144a97c2 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:22:32.504929 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 747/1497] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:22:32 2022] POST /identity/v3/projects => generated 379 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:22:32.509694 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33df32-cda2-4315-bdb1-cba3a4c3c357 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:22:32.511282 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-3a33df32-cda2-4315-bdb1-cba3a4c3c357 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:22:32.546271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33df32-cda2-4315-bdb1-cba3a4c3c357 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:22:32.638037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef161e-58c3-4fbf-b16c-0c5f4d717d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:22:32.638631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef161e-58c3-4fbf-b16c-0c5f4d717d9b admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:22:32.638945 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef161e-58c3-4fbf-b16c-0c5f4d717d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:22:32.639216 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef161e-58c3-4fbf-b16c-0c5f4d717d9b admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:22:32.639827 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aef161e-58c3-4fbf-b16c-0c5f4d717d9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:22:32.640831 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: WARNING py.warnings [None req-1aef161e-58c3-4fbf-b16c-0c5f4d717d9b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 14:22:32.640831 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: warnings.warn(msg) Sep 24 14:22:32.640831 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: Sep 24 14:22:32.642024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aef161e-58c3-4fbf-b16c-0c5f4d717d9b admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:22:32.684951 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 751/1498] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:22:32 2022] POST /identity/v3/users => generated 290 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:22:32.689071 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4271e011-2ee7-449b-bff5-5932144a97c2 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:22:32.690668 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-4271e011-2ee7-449b-bff5-5932144a97c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:22:32.752025 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4271e011-2ee7-449b-bff5-5932144a97c2 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:22:32.833812 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:22:32.845010 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae7874c-f418-41b4-a0ef-08255154b9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:22:32.845609 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7874c-f418-41b4-a0ef-08255154b9bb admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:22:32.845881 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7874c-f418-41b4-a0ef-08255154b9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:22:32.846146 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7874c-f418-41b4-a0ef-08255154b9bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:22:32.846775 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae7874c-f418-41b4-a0ef-08255154b9bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:22:32.847804 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: WARNING py.warnings [None req-6ae7874c-f418-41b4-a0ef-08255154b9bb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 14:22:32.847804 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: warnings.warn(msg) Sep 24 14:22:32.847804 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: Sep 24 14:22:32.848726 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae7874c-f418-41b4-a0ef-08255154b9bb admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:22:32.861326 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 748/1499] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:22:32 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:22:32.865286 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef161e-58c3-4fbf-b16c-0c5f4d717d9b admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:22:32.866768 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-1aef161e-58c3-4fbf-b16c-0c5f4d717d9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:22:32.878394 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef161e-58c3-4fbf-b16c-0c5f4d717d9b admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:22:32.902342 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:22:32.972661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2850a657-716b-4772-ae17-8084d87f6c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:22:32.973261 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850a657-716b-4772-ae17-8084d87f6c22 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:22:32.973529 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850a657-716b-4772-ae17-8084d87f6c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:22:32.973789 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850a657-716b-4772-ae17-8084d87f6c22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:22:32.974421 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2850a657-716b-4772-ae17-8084d87f6c22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:22:32.975462 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: WARNING py.warnings [None req-2850a657-716b-4772-ae17-8084d87f6c22 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 14:22:32.975462 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: warnings.warn(msg) Sep 24 14:22:32.975462 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: Sep 24 14:22:32.976334 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2850a657-716b-4772-ae17-8084d87f6c22 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:22:32.989483 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 752/1500] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:22:32 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:22:32.994352 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae7874c-f418-41b4-a0ef-08255154b9bb admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:22:32.995942 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-6ae7874c-f418-41b4-a0ef-08255154b9bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:22:33.030312 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae7874c-f418-41b4-a0ef-08255154b9bb admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:22:33.123476 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f28278-1473-48a0-9e01-8110edf6fea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:22:33.124066 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f28278-1473-48a0-9e01-8110edf6fea6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:22:33.124343 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f28278-1473-48a0-9e01-8110edf6fea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:22:33.124613 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f28278-1473-48a0-9e01-8110edf6fea6 admin admin] PATH_INFO: `/v3/projects/0901e5a9ef6343e98a708f1de1813c79/users/cb7002716f694a08b29da3b37f7dbbfe/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:22:33.146697 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f28278-1473-48a0-9e01-8110edf6fea6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0901e5a9ef6343e98a708f1de1813c79, user_id=cb7002716f694a08b29da3b37f7dbbfe, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:22:33.147754 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: WARNING py.warnings [None req-87f28278-1473-48a0-9e01-8110edf6fea6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 14:22:33.147754 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: warnings.warn(msg) Sep 24 14:22:33.147754 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: Sep 24 14:22:33.150223 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f28278-1473-48a0-9e01-8110edf6fea6 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:22:33.174797 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 749/1501] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:22:32 2022] PUT /identity/v3/projects/0901e5a9ef6343e98a708f1de1813c79/users/cb7002716f694a08b29da3b37f7dbbfe/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:22:33.186865 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb6488-fd58-45c6-b70c-f687905e246a None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:22:33.187151 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb6488-fd58-45c6-b70c-f687905e246a None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:22:33.187422 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb6488-fd58-45c6-b70c-f687905e246a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:22:33.243366 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-6bfb6488-fd58-45c6-b70c-f687905e246a None None] MFA Rules not processed for user `cb7002716f694a08b29da3b37f7dbbfe`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:22:33.244555 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-6bfb6488-fd58-45c6-b70c-f687905e246a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:22:33.337962 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 753/1502] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:22:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:22:33.351130 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b4c5d-a471-4ade-a6ba-eb113b61d43e None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:22:33.351420 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b4c5d-a471-4ade-a6ba-eb113b61d43e None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:22:33.352269 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b4c5d-a471-4ade-a6ba-eb113b61d43e None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:22:33.353593 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 750/1503] 10.222.0.52 () {54 vars in 963 bytes} [Sat Sep 24 14:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:22:33.358260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c927aa-ea83-4f1e-931b-b2e65c566549 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:22:33.358546 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c927aa-ea83-4f1e-931b-b2e65c566549 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:22:33.358835 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c927aa-ea83-4f1e-931b-b2e65c566549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:22:33.389086 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:22:33.433806 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-e8c927aa-ea83-4f1e-931b-b2e65c566549 None None] MFA Rules not processed for user `63921623056045a9b97b2de3ca418138`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:22:33.435014 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-e8c927aa-ea83-4f1e-931b-b2e65c566549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:22:33.508867 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 754/1504] 10.222.0.52 () {58 vars in 1060 bytes} [Sat Sep 24 14:22:33 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:22:33.518482 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b4c5d-a471-4ade-a6ba-eb113b61d43e None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:22:33.520074 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-b51b4c5d-a471-4ade-a6ba-eb113b61d43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:22:33.570954 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:22:33.624772 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b4c5d-a471-4ade-a6ba-eb113b61d43e None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:22:33.697563 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc4d45-252a-4892-a741-c758bca73faa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:22:33.698079 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc4d45-252a-4892-a741-c758bca73faa service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:22:33.698356 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc4d45-252a-4892-a741-c758bca73faa service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:22:33.698623 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc4d45-252a-4892-a741-c758bca73faa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:22:33.700248 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-10cc4d45-252a-4892-a741-c758bca73faa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:22:33.797786 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cc4d45-252a-4892-a741-c758bca73faa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:22:33.799278 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cc4d45-252a-4892-a741-c758bca73faa service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:22:33.802703 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:22:33.818325 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 751/1505] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:22:33 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:22:33.863172 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462264d4-3f14-4d6e-a4ea-356793c57b85 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:22:33.863486 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462264d4-3f14-4d6e-a4ea-356793c57b85 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:22:33.863766 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462264d4-3f14-4d6e-a4ea-356793c57b85 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:22:33.865033 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 755/1506] 10.222.0.52 () {52 vars in 830 bytes} [Sat Sep 24 14:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:22:33.869241 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc4d45-252a-4892-a741-c758bca73faa service cinder] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:22:33.870451 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-10cc4d45-252a-4892-a741-c758bca73faa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:22:33.971193 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc4d45-252a-4892-a741-c758bca73faa service cinder] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:22:34.062460 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47548e16-163a-4809-b0fa-b43b548767b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:22:34.063171 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47548e16-163a-4809-b0fa-b43b548767b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:22:34.063418 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47548e16-163a-4809-b0fa-b43b548767b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:22:34.063720 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47548e16-163a-4809-b0fa-b43b548767b2 admin admin] PATH_INFO: `/v3/users/cb7002716f694a08b29da3b37f7dbbfe` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:22:34.077178 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47548e16-163a-4809-b0fa-b43b548767b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb7002716f694a08b29da3b37f7dbbfe)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:22:34.078201 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: WARNING py.warnings [None req-47548e16-163a-4809-b0fa-b43b548767b2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 14:22:34.078201 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: warnings.warn(msg) Sep 24 14:22:34.078201 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: Sep 24 14:22:34.079446 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47548e16-163a-4809-b0fa-b43b548767b2 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:22:34.089052 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:22:34.143907 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-47548e16-163a-4809-b0fa-b43b548767b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb7002716f694a08b29da3b37f7dbbfe'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:22:34.144442 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-47548e16-163a-4809-b0fa-b43b548767b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb7002716f694a08b29da3b37f7dbbfe'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:22:34.144865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-47548e16-163a-4809-b0fa-b43b548767b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb7002716f694a08b29da3b37f7dbbfe'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:22:34.167269 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-47548e16-163a-4809-b0fa-b43b548767b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb7002716f694a08b29da3b37f7dbbfe'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:22:34.181378 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-47548e16-163a-4809-b0fa-b43b548767b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb7002716f694a08b29da3b37f7dbbfe'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:22:34.196243 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 752/1507] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:22:33 2022] DELETE /identity/v3/users/cb7002716f694a08b29da3b37f7dbbfe => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:22:34.200619 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a672e-c899-4766-9ebb-9fcf13c15e92 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:22:34.201089 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a672e-c899-4766-9ebb-9fcf13c15e92 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:22:34.201197 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a672e-c899-4766-9ebb-9fcf13c15e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:22:34.256628 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-9d7a672e-c899-4766-9ebb-9fcf13c15e92 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:22:34.258328 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-9d7a672e-c899-4766-9ebb-9fcf13c15e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:22:34.336752 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:22:34.349334 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 756/1508] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:22:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:22:34.359351 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47548e16-163a-4809-b0fa-b43b548767b2 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:22:34.360572 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-47548e16-163a-4809-b0fa-b43b548767b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:22:34.458752 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47548e16-163a-4809-b0fa-b43b548767b2 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:22:34.516918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:22:34.537581 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52176e90-5966-425d-b15c-c2cfe14c8981 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:22:34.538198 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52176e90-5966-425d-b15c-c2cfe14c8981 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:22:34.538469 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52176e90-5966-425d-b15c-c2cfe14c8981 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:22:34.538738 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52176e90-5966-425d-b15c-c2cfe14c8981 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:22:34.540844 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-52176e90-5966-425d-b15c-c2cfe14c8981 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:22:34.654648 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52176e90-5966-425d-b15c-c2cfe14c8981 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:22:34.656107 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52176e90-5966-425d-b15c-c2cfe14c8981 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:22:34.674613 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 753/1509] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:22:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:22:34.992918 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a672e-c899-4766-9ebb-9fcf13c15e92 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:22:34.994140 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-9d7a672e-c899-4766-9ebb-9fcf13c15e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:22:35.005844 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a672e-c899-4766-9ebb-9fcf13c15e92 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:22:35.102653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a11d9-51d0-4fcb-9faf-dafa81ef4478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:22:35.103326 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a11d9-51d0-4fcb-9faf-dafa81ef4478 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:22:35.103601 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a11d9-51d0-4fcb-9faf-dafa81ef4478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:22:35.103865 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a11d9-51d0-4fcb-9faf-dafa81ef4478 admin admin] PATH_INFO: `/v3/projects/0901e5a9ef6343e98a708f1de1813c79` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:22:35.113628 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4a11d9-51d0-4fcb-9faf-dafa81ef4478 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0901e5a9ef6343e98a708f1de1813c79)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:22:35.114661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: WARNING py.warnings [None req-0e4a11d9-51d0-4fcb-9faf-dafa81ef4478 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 14:22:35.114661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: warnings.warn(msg) Sep 24 14:22:35.114661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: Sep 24 14:22:35.115882 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4a11d9-51d0-4fcb-9faf-dafa81ef4478 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:22:35.175076 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-0e4a11d9-51d0-4fcb-9faf-dafa81ef4478 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0901e5a9ef6343e98a708f1de1813c79', 'request_id': 'req-0e4a11d9-51d0-4fcb-9faf-dafa81ef4478'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:22:35.188064 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-0e4a11d9-51d0-4fcb-9faf-dafa81ef4478 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0901e5a9ef6343e98a708f1de1813c79', 'request_id': 'req-0e4a11d9-51d0-4fcb-9faf-dafa81ef4478'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:22:35.194050 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-0e4a11d9-51d0-4fcb-9faf-dafa81ef4478 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0901e5a9ef6343e98a708f1de1813c79', 'request_id': 'req-0e4a11d9-51d0-4fcb-9faf-dafa81ef4478'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:22:35.220342 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-0e4a11d9-51d0-4fcb-9faf-dafa81ef4478 admin admin] The token cache is being invalidate because project 0901e5a9ef6343e98a708f1de1813c79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:22:35.220971 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-0e4a11d9-51d0-4fcb-9faf-dafa81ef4478 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:22:35.221068 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-0e4a11d9-51d0-4fcb-9faf-dafa81ef4478 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:22:35.222448 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 757/1510] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:22:34 2022] DELETE /identity/v3/projects/0901e5a9ef6343e98a708f1de1813c79 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:22:44.672454 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=62 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:22:45.218241 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:24:06.762035 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b3a7e-2970-412c-a881-ad9b649d2a70 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:24:06.762630 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b3a7e-2970-412c-a881-ad9b649d2a70 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:24:06.762630 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b3a7e-2970-412c-a881-ad9b649d2a70 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:24:06.763981 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 754/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 14:24:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:24:06.768911 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ca11e-66bc-42eb-9b46-e2ffbbf6e51d None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:24:06.769337 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ca11e-66bc-42eb-9b46-e2ffbbf6e51d None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:24:06.769463 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ca11e-66bc-42eb-9b46-e2ffbbf6e51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:24:06.850845 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-777ca11e-66bc-42eb-9b46-e2ffbbf6e51d None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:24:06.852712 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-777ca11e-66bc-42eb-9b46-e2ffbbf6e51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:24:06.952098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 758/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 14:24:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:24:07.107385 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8b3a7e-2970-412c-a881-ad9b649d2a70 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:24:07.109469 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-ef8b3a7e-2970-412c-a881-ad9b649d2a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:24:07.226127 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8b3a7e-2970-412c-a881-ad9b649d2a70 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:24:07.231691 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eac753-2bb9-4cdc-a82d-33f1fe4e4a05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:24:07.232300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eac753-2bb9-4cdc-a82d-33f1fe4e4a05 service nova] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:24:07.232578 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eac753-2bb9-4cdc-a82d-33f1fe4e4a05 service nova] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:24:07.232851 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eac753-2bb9-4cdc-a82d-33f1fe4e4a05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:24:07.240852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eac753-2bb9-4cdc-a82d-33f1fe4e4a05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:24:07.242427 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eac753-2bb9-4cdc-a82d-33f1fe4e4a05 service nova] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:24:07.251699 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 755/1513] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:24:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:24:08.334960 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a07d55-1a7e-4859-ad27-f453f87ea8ea None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:24:08.335399 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a07d55-1a7e-4859-ad27-f453f87ea8ea None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:24:08.335717 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a07d55-1a7e-4859-ad27-f453f87ea8ea None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:24:08.338525 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 759/1514] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 14:24:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:24:08.345235 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd5b40-4e40-440a-8651-5ee3f544891c None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:24:08.345740 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd5b40-4e40-440a-8651-5ee3f544891c None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:24:08.345853 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd5b40-4e40-440a-8651-5ee3f544891c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:24:08.365503 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-63cd5b40-4e40-440a-8651-5ee3f544891c None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:24:08.366767 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-63cd5b40-4e40-440a-8651-5ee3f544891c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:24:08.374663 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 756/1515] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 14:24:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:24:08.482308 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e107d-4188-4ce9-a090-0c60e81ad903 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:24:08.482660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e107d-4188-4ce9-a090-0c60e81ad903 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:24:08.483023 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e107d-4188-4ce9-a090-0c60e81ad903 None None] PATH_INFO: `/v3` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:24:08.484367 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 760/1516] 10.222.0.33 () {54 vars in 925 bytes} [Sat Sep 24 14:24:08 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:24:08.490553 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5f889-36f4-44ca-bf19-fc79d36669ab None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:24:08.490857 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5f889-36f4-44ca-bf19-fc79d36669ab None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:24:08.491141 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5f889-36f4-44ca-bf19-fc79d36669ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:24:08.536367 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3e107d-4188-4ce9-a090-0c60e81ad903 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:24:08.553134 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3e107d-4188-4ce9-a090-0c60e81ad903 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:24:08.555943 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-d0e5f889-36f4-44ca-bf19-fc79d36669ab None None] MFA Rules not processed for user `18fe384ecd454fb298dcc788f41dd44b`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:24:08.557145 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-d0e5f889-36f4-44ca-bf19-fc79d36669ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:24:08.560292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe86ea6-9658-419f-86bb-ccc77926f509 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:24:08.560912 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe86ea6-9658-419f-86bb-ccc77926f509 service nova] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:24:08.561196 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe86ea6-9658-419f-86bb-ccc77926f509 service nova] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:24:08.561469 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe86ea6-9658-419f-86bb-ccc77926f509 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:24:08.570580 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe86ea6-9658-419f-86bb-ccc77926f509 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:24:08.572193 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe86ea6-9658-419f-86bb-ccc77926f509 service nova] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:24:08.581846 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 761/1517] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:24:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 14:24:08.612178 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:24:08.635010 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 757/1518] 10.222.0.33 () {58 vars in 1015 bytes} [Sat Sep 24 14:24:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:24:17.681805 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89249f43-33f5-4578-8b29-d3abd23932e1 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:24:17.682248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89249f43-33f5-4578-8b29-d3abd23932e1 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:24:17.682373 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89249f43-33f5-4578-8b29-d3abd23932e1 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:24:17.683727 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 762/1519] 10.222.0.52 () {54 vars in 961 bytes} [Sat Sep 24 14:24:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:24:17.688593 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e41a7e-095f-48f0-b516-e9526d8535f9 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:24:17.689048 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e41a7e-095f-48f0-b516-e9526d8535f9 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:24:17.689146 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e41a7e-095f-48f0-b516-e9526d8535f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:24:17.708206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-d4e41a7e-095f-48f0-b516-e9526d8535f9 None None] MFA Rules not processed for user `18fe384ecd454fb298dcc788f41dd44b`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:24:17.709378 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-d4e41a7e-095f-48f0-b516-e9526d8535f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:24:17.718267 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 758/1520] 10.222.0.52 () {58 vars in 1058 bytes} [Sat Sep 24 14:24:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:24:17.726082 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89249f43-33f5-4578-8b29-d3abd23932e1 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:24:17.739853 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89249f43-33f5-4578-8b29-d3abd23932e1 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:24:17.745416 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7203f24e-766e-4639-bef5-625c88daaa77 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18fe384ecd454fb298dcc788f41dd44b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:24:17.746008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203f24e-766e-4639-bef5-625c88daaa77 service placement] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:24:17.746285 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203f24e-766e-4639-bef5-625c88daaa77 service placement] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:24:17.746553 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203f24e-766e-4639-bef5-625c88daaa77 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:24:17.754613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7203f24e-766e-4639-bef5-625c88daaa77 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:24:17.756225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7203f24e-766e-4639-bef5-625c88daaa77 service placement] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:24:17.765654 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 763/1521] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:24:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:24:20.050299 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8092024-24a8-453e-a066-088a79987298 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:24:20.050917 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8092024-24a8-453e-a066-088a79987298 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:24:20.050917 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8092024-24a8-453e-a066-088a79987298 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:24:20.052240 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 759/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 14:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:24:20.059069 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c57c2b-beb2-4578-839f-535494bb3199 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:24:20.059509 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c57c2b-beb2-4578-839f-535494bb3199 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:24:20.059613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c57c2b-beb2-4578-839f-535494bb3199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:24:20.078906 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-f6c57c2b-beb2-4578-839f-535494bb3199 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:24:20.080090 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-f6c57c2b-beb2-4578-839f-535494bb3199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:24:20.087741 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 764/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 14:24:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:24:20.243470 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8092024-24a8-453e-a066-088a79987298 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:24:20.257781 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8092024-24a8-453e-a066-088a79987298 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:24:20.263219 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f9c7eb-fed2-4820-a437-65157f5ff285 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:24:20.263803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9c7eb-fed2-4820-a437-65157f5ff285 service nova] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:24:20.264075 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9c7eb-fed2-4820-a437-65157f5ff285 service nova] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:24:20.264343 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9c7eb-fed2-4820-a437-65157f5ff285 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:24:20.272230 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f9c7eb-fed2-4820-a437-65157f5ff285 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:24:20.273795 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f9c7eb-fed2-4820-a437-65157f5ff285 service nova] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:24:20.283345 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 760/1524] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:24:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:24:21.356525 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65055c14-6de9-4a84-bc75-f9945dc658d3 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:24:21.357090 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65055c14-6de9-4a84-bc75-f9945dc658d3 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:24:21.357090 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65055c14-6de9-4a84-bc75-f9945dc658d3 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:24:21.358386 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 765/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 14:24:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:24:21.364889 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36390999-a497-4351-ae25-4d9eb16ff1ce None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:24:21.365371 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36390999-a497-4351-ae25-4d9eb16ff1ce None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:24:21.365479 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36390999-a497-4351-ae25-4d9eb16ff1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:24:21.385682 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-36390999-a497-4351-ae25-4d9eb16ff1ce None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:24:21.386927 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-36390999-a497-4351-ae25-4d9eb16ff1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:24:21.399167 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 761/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 14:24:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:24:21.546515 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65055c14-6de9-4a84-bc75-f9945dc658d3 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:24:21.561027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65055c14-6de9-4a84-bc75-f9945dc658d3 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:24:21.566359 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941fe5e8-2819-4ada-9e87-2832aa693059 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:24:21.566969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941fe5e8-2819-4ada-9e87-2832aa693059 service nova] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:24:21.567245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941fe5e8-2819-4ada-9e87-2832aa693059 service nova] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:24:21.567510 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941fe5e8-2819-4ada-9e87-2832aa693059 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:24:21.575370 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941fe5e8-2819-4ada-9e87-2832aa693059 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:24:21.576939 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941fe5e8-2819-4ada-9e87-2832aa693059 service nova] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:24:21.586130 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 766/1527] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:24:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 14:24:25.676232 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053ff57-4e27-4951-8f11-27e62d7ef458 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:24:25.676747 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053ff57-4e27-4951-8f11-27e62d7ef458 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:24:25.676845 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053ff57-4e27-4951-8f11-27e62d7ef458 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:24:25.678119 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 762/1528] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 14:24:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:24:25.683384 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72e35f-fdea-4635-8ac7-133d3a850c10 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:24:25.683797 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72e35f-fdea-4635-8ac7-133d3a850c10 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:24:25.683911 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72e35f-fdea-4635-8ac7-133d3a850c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:24:25.703011 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-4e72e35f-fdea-4635-8ac7-133d3a850c10 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:24:25.704225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-4e72e35f-fdea-4635-8ac7-133d3a850c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:24:25.711926 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 767/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 14:24:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:24:25.998388 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e053ff57-4e27-4951-8f11-27e62d7ef458 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:24:26.000143 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-e053ff57-4e27-4951-8f11-27e62d7ef458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:24:26.078761 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e053ff57-4e27-4951-8f11-27e62d7ef458 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:24:26.082372 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75da3caf-b4f9-4190-a90d-485d902fb9e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:24:26.083003 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da3caf-b4f9-4190-a90d-485d902fb9e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:24:26.083282 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da3caf-b4f9-4190-a90d-485d902fb9e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:24:26.083553 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da3caf-b4f9-4190-a90d-485d902fb9e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:24:26.092899 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75da3caf-b4f9-4190-a90d-485d902fb9e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:24:26.094306 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75da3caf-b4f9-4190-a90d-485d902fb9e8 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:24:26.103540 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 763/1530] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:24:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:24:28.274430 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb5c89f-699f-4a41-9a40-09192eddfacb None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:24:28.274939 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb5c89f-699f-4a41-9a40-09192eddfacb None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:24:28.275057 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb5c89f-699f-4a41-9a40-09192eddfacb None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:24:28.277013 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 768/1531] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 14:24:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:24:28.282250 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ffe9a9-5031-49fb-a6c5-a9c3b16ff93c None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:24:28.282648 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ffe9a9-5031-49fb-a6c5-a9c3b16ff93c None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:24:28.282833 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ffe9a9-5031-49fb-a6c5-a9c3b16ff93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:24:28.304216 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-58ffe9a9-5031-49fb-a6c5-a9c3b16ff93c None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:24:28.305437 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-58ffe9a9-5031-49fb-a6c5-a9c3b16ff93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:24:28.313391 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 764/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 14:24:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:24:28.609426 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb5c89f-699f-4a41-9a40-09192eddfacb None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:24:28.611140 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-ccb5c89f-699f-4a41-9a40-09192eddfacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:24:28.624617 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb5c89f-699f-4a41-9a40-09192eddfacb None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:24:28.628161 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1675a95-5565-4e69-ab68-8077d306ab89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:24:28.628755 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1675a95-5565-4e69-ab68-8077d306ab89 service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:24:28.629036 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1675a95-5565-4e69-ab68-8077d306ab89 service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:24:28.629309 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1675a95-5565-4e69-ab68-8077d306ab89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:24:28.638638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1675a95-5565-4e69-ab68-8077d306ab89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:24:28.640114 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1675a95-5565-4e69-ab68-8077d306ab89 service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:24:28.652506 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 769/1533] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:24:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:24:32.352212 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475afc8e-8718-4127-bde2-4e3196201416 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:24:32.352872 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475afc8e-8718-4127-bde2-4e3196201416 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:24:32.352872 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475afc8e-8718-4127-bde2-4e3196201416 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:24:32.354086 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 765/1534] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 14:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:24:32.359457 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e281ffb-b854-4573-946c-ce6421086641 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:24:32.359893 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e281ffb-b854-4573-946c-ce6421086641 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:24:32.360006 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e281ffb-b854-4573-946c-ce6421086641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:24:32.367496 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:24:32.380645 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-7e281ffb-b854-4573-946c-ce6421086641 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:24:32.381842 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-7e281ffb-b854-4573-946c-ce6421086641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:24:32.390216 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 770/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 14:24:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:24:32.696272 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475afc8e-8718-4127-bde2-4e3196201416 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:24:32.698548 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-475afc8e-8718-4127-bde2-4e3196201416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:24:32.774560 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475afc8e-8718-4127-bde2-4e3196201416 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:24:32.778577 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bb5544-dcaf-4576-8d15-44e9c0f74ef4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:24:32.779201 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb5544-dcaf-4576-8d15-44e9c0f74ef4 service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:24:32.779477 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb5544-dcaf-4576-8d15-44e9c0f74ef4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:24:32.779745 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb5544-dcaf-4576-8d15-44e9c0f74ef4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:24:32.788541 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bb5544-dcaf-4576-8d15-44e9c0f74ef4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:24:32.789949 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bb5544-dcaf-4576-8d15-44e9c0f74ef4 service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:24:32.799282 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 766/1536] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:24:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:24:34.943889 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2fa074-cdfb-422d-8a3b-da6d227836da None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:24:34.944444 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2fa074-cdfb-422d-8a3b-da6d227836da None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:24:34.944444 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2fa074-cdfb-422d-8a3b-da6d227836da None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:24:34.945785 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 771/1537] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 14:24:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:24:34.951088 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af22a24-c3c1-4f33-8f0a-b486898f2922 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:24:34.951486 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af22a24-c3c1-4f33-8f0a-b486898f2922 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:24:34.951692 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af22a24-c3c1-4f33-8f0a-b486898f2922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:24:34.970780 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-9af22a24-c3c1-4f33-8f0a-b486898f2922 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:24:34.971996 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-9af22a24-c3c1-4f33-8f0a-b486898f2922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:24:34.980201 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 767/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 14:24:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:24:35.279441 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2fa074-cdfb-422d-8a3b-da6d227836da None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:24:35.285418 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-3f2fa074-cdfb-422d-8a3b-da6d227836da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:24:35.294689 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2fa074-cdfb-422d-8a3b-da6d227836da None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:24:35.298077 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fdf901-7a57-4430-a9fa-7cbffd22716d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:24:35.298663 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fdf901-7a57-4430-a9fa-7cbffd22716d service cinder] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:24:35.298967 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fdf901-7a57-4430-a9fa-7cbffd22716d service cinder] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:24:35.299237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fdf901-7a57-4430-a9fa-7cbffd22716d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:24:35.307909 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fdf901-7a57-4430-a9fa-7cbffd22716d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:24:35.309319 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fdf901-7a57-4430-a9fa-7cbffd22716d service cinder] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:24:35.318435 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 772/1539] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:24:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:24:44.965938 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=92 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:24:45.314360 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=8 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:01.285972 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a11c35f-aa42-4412-871d-246e1202d2d2 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:01.286656 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a11c35f-aa42-4412-871d-246e1202d2d2 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:01.286656 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a11c35f-aa42-4412-871d-246e1202d2d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:01.313801 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-9a11c35f-aa42-4412-871d-246e1202d2d2 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:01.316763 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-9a11c35f-aa42-4412-871d-246e1202d2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:01.331777 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 768/1540] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:01.343426 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe09710-2a64-4e97-8330-d87f164b8c66 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:01.344029 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe09710-2a64-4e97-8330-d87f164b8c66 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:01.344029 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe09710-2a64-4e97-8330-d87f164b8c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:01.373336 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-abe09710-2a64-4e97-8330-d87f164b8c66 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:01.374562 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-abe09710-2a64-4e97-8330-d87f164b8c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:01.392079 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 773/1541] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:01.397821 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a11c35f-aa42-4412-871d-246e1202d2d2 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:01.410375 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a11c35f-aa42-4412-871d-246e1202d2d2 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:01.421734 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350805fb-9039-49c9-92df-5b3a34f63168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:01.422751 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350805fb-9039-49c9-92df-5b3a34f63168 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:01.423356 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350805fb-9039-49c9-92df-5b3a34f63168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:01.423855 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350805fb-9039-49c9-92df-5b3a34f63168 admin admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:01.424738 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350805fb-9039-49c9-92df-5b3a34f63168 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:26:01.425293 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350805fb-9039-49c9-92df-5b3a34f63168 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:01.426803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: WARNING py.warnings [None req-350805fb-9039-49c9-92df-5b3a34f63168 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 14:26:01.426803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: warnings.warn(msg) Sep 24 14:26:01.426803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: Sep 24 14:26:01.428347 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350805fb-9039-49c9-92df-5b3a34f63168 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:01.443805 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 769/1542] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:26:01.453455 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe09710-2a64-4e97-8330-d87f164b8c66 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:01.467998 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe09710-2a64-4e97-8330-d87f164b8c66 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:01.479798 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36d0b90-0039-43ab-a9fd-abbd660be184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:01.480416 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d0b90-0039-43ab-a9fd-abbd660be184 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:01.480692 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d0b90-0039-43ab-a9fd-abbd660be184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:01.480959 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d0b90-0039-43ab-a9fd-abbd660be184 admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:01.481591 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36d0b90-0039-43ab-a9fd-abbd660be184 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:01.482621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: WARNING py.warnings [None req-a36d0b90-0039-43ab-a9fd-abbd660be184 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 14:26:01.482621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: warnings.warn(msg) Sep 24 14:26:01.482621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: Sep 24 14:26:01.484450 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36d0b90-0039-43ab-a9fd-abbd660be184 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:01.490461 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81170b2f-9425-476f-b8a5-34bb5e081bd1 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:01.491266 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81170b2f-9425-476f-b8a5-34bb5e081bd1 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:01.491783 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81170b2f-9425-476f-b8a5-34bb5e081bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:01.524613 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-81170b2f-9425-476f-b8a5-34bb5e081bd1 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:01.526355 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-81170b2f-9425-476f-b8a5-34bb5e081bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:01.540545 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 770/1543] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:01.549949 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7323a38a-fa26-4659-9e7c-77dcbb0e8ce5 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:01.550563 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7323a38a-fa26-4659-9e7c-77dcbb0e8ce5 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:01.551081 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7323a38a-fa26-4659-9e7c-77dcbb0e8ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:01.552009 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 774/1544] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:26:01 2022] POST /identity/v3/projects => generated 355 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:01.560986 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36d0b90-0039-43ab-a9fd-abbd660be184 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:01.593178 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-7323a38a-fa26-4659-9e7c-77dcbb0e8ce5 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:01.595175 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-7323a38a-fa26-4659-9e7c-77dcbb0e8ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:01.698291 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36d0b90-0039-43ab-a9fd-abbd660be184 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:01.711045 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0175255-a95d-4428-8018-134403cf90b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:01.711711 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0175255-a95d-4428-8018-134403cf90b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:01.711988 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0175255-a95d-4428-8018-134403cf90b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:01.712255 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0175255-a95d-4428-8018-134403cf90b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:01.712894 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0175255-a95d-4428-8018-134403cf90b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:01.713272 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 771/1545] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:26:01.715149 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0175255-a95d-4428-8018-134403cf90b3 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:01.715845 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca09df-9fea-46ee-aebc-120ac22e9a47 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:01.716378 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca09df-9fea-46ee-aebc-120ac22e9a47 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:01.716871 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca09df-9fea-46ee-aebc-120ac22e9a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:01.741977 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-75ca09df-9fea-46ee-aebc-120ac22e9a47 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:01.744274 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-75ca09df-9fea-46ee-aebc-120ac22e9a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:01.749356 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 775/1546] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:26:01 2022] POST /identity/v3/users => generated 278 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:01.749356 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f106d16-490c-424b-8b55-71ae3c0bfd31 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:01.749356 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f106d16-490c-424b-8b55-71ae3c0bfd31 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:01.749356 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f106d16-490c-424b-8b55-71ae3c0bfd31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:01.758575 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 772/1547] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:01.760046 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2353fd-f991-4bd7-ad09-11f32f8a1083 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:01.760334 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2353fd-f991-4bd7-ad09-11f32f8a1083 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:01.760610 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2353fd-f991-4bd7-ad09-11f32f8a1083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:01.772423 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-0f106d16-490c-424b-8b55-71ae3c0bfd31 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:01.773712 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-0f106d16-490c-424b-8b55-71ae3c0bfd31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:01.782081 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 776/1548] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:01.783373 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-2c2353fd-f991-4bd7-ad09-11f32f8a1083 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:01.784917 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-2c2353fd-f991-4bd7-ad09-11f32f8a1083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:01.787368 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd16a6-4abf-40ac-9fcf-4ffde782dc0e None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:01.788103 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd16a6-4abf-40ac-9fcf-4ffde782dc0e None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:01.788777 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd16a6-4abf-40ac-9fcf-4ffde782dc0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:01.795200 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 773/1549] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:01.800335 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2353fd-f991-4bd7-ad09-11f32f8a1083 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:01.811913 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2353fd-f991-4bd7-ad09-11f32f8a1083 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:01.812566 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-e9bd16a6-4abf-40ac-9fcf-4ffde782dc0e None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:01.813784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-e9bd16a6-4abf-40ac-9fcf-4ffde782dc0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:01.823300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c993914-cbf7-4ad5-b55d-15ca9356d36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:01.824548 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c993914-cbf7-4ad5-b55d-15ca9356d36f admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:01.824896 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c993914-cbf7-4ad5-b55d-15ca9356d36f admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:01.825257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c993914-cbf7-4ad5-b55d-15ca9356d36f admin admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:01.826213 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c993914-cbf7-4ad5-b55d-15ca9356d36f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:26:01.826586 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c993914-cbf7-4ad5-b55d-15ca9356d36f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:01.828304 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c993914-cbf7-4ad5-b55d-15ca9356d36f admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:01.828850 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 777/1550] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:01.830865 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd16a6-4abf-40ac-9fcf-4ffde782dc0e None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:01.845441 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd16a6-4abf-40ac-9fcf-4ffde782dc0e None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:01.846850 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 774/1551] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:26:01.847620 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c711274-ddce-4b65-8625-d7e2f0c1d68a None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:01.847949 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c711274-ddce-4b65-8625-d7e2f0c1d68a None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:01.848167 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c711274-ddce-4b65-8625-d7e2f0c1d68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:01.850341 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2da9b33-e856-4014-a52b-567eff9f0a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:01.850983 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da9b33-e856-4014-a52b-567eff9f0a0e admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:01.851277 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da9b33-e856-4014-a52b-567eff9f0a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:01.851539 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da9b33-e856-4014-a52b-567eff9f0a0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:01.852872 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2da9b33-e856-4014-a52b-567eff9f0a0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:01.854555 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2da9b33-e856-4014-a52b-567eff9f0a0e admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:01.867732 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 778/1552] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:01.867732 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5f892-a420-475f-bdae-9d5c09e420c5 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:01.867732 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5f892-a420-475f-bdae-9d5c09e420c5 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:01.867732 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5f892-a420-475f-bdae-9d5c09e420c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:01.868263 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-0c711274-ddce-4b65-8625-d7e2f0c1d68a None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:01.869380 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-0c711274-ddce-4b65-8625-d7e2f0c1d68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:01.882867 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 775/1553] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:01.882867 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a80c8-8c38-4b96-b659-0d887142cd25 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:01.882867 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a80c8-8c38-4b96-b659-0d887142cd25 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:01.882867 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a80c8-8c38-4b96-b659-0d887142cd25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:01.886410 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-f4e5f892-a420-475f-bdae-9d5c09e420c5 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:01.887651 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-f4e5f892-a420-475f-bdae-9d5c09e420c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:01.895947 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 779/1554] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:01.897187 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4cf08-511e-46d1-ad4a-56d2023d42c8 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:01.897472 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4cf08-511e-46d1-ad4a-56d2023d42c8 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:01.897746 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4cf08-511e-46d1-ad4a-56d2023d42c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:01.900300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-0f3a80c8-8c38-4b96-b659-0d887142cd25 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:01.901475 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-0f3a80c8-8c38-4b96-b659-0d887142cd25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:01.909536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 776/1555] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:01.912130 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff489cf-4a27-4df1-bedf-d84e215f0888 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:01.912532 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff489cf-4a27-4df1-bedf-d84e215f0888 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:01.912817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff489cf-4a27-4df1-bedf-d84e215f0888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:01.919152 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-c0c4cf08-511e-46d1-ad4a-56d2023d42c8 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:01.920355 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-c0c4cf08-511e-46d1-ad4a-56d2023d42c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:01.928316 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 780/1556] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:01.929463 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d015cdc-3d87-4ab4-bc4d-c609bbcf3b9a None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:01.929754 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d015cdc-3d87-4ab4-bc4d-c609bbcf3b9a None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:01.930065 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d015cdc-3d87-4ab4-bc4d-c609bbcf3b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:01.934716 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-eff489cf-4a27-4df1-bedf-d84e215f0888 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:01.936058 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-eff489cf-4a27-4df1-bedf-d84e215f0888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:01.943860 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 777/1557] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:01.944604 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff489cf-4a27-4df1-bedf-d84e215f0888 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:01.949266 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-5d015cdc-3d87-4ab4-bc4d-c609bbcf3b9a None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:01.950449 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-5d015cdc-3d87-4ab4-bc4d-c609bbcf3b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:01.955821 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff489cf-4a27-4df1-bedf-d84e215f0888 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:01.958528 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 781/1558] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:01.959389 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d015cdc-3d87-4ab4-bc4d-c609bbcf3b9a None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:01.964018 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49171fa-9ab9-4068-87b2-93eb7d81b2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:01.964613 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49171fa-9ab9-4068-87b2-93eb7d81b2d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:01.964889 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49171fa-9ab9-4068-87b2-93eb7d81b2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:01.965159 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49171fa-9ab9-4068-87b2-93eb7d81b2d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:01.965785 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49171fa-9ab9-4068-87b2-93eb7d81b2d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:01.967734 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49171fa-9ab9-4068-87b2-93eb7d81b2d9 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:01.971124 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d015cdc-3d87-4ab4-bc4d-c609bbcf3b9a None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:01.975970 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f929cd6a-bbac-41db-9c5f-adfa24bcc5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:01.976561 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929cd6a-bbac-41db-9c5f-adfa24bcc5b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:01.977405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929cd6a-bbac-41db-9c5f-adfa24bcc5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:01.977679 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929cd6a-bbac-41db-9c5f-adfa24bcc5b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:01.978322 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f929cd6a-bbac-41db-9c5f-adfa24bcc5b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:01.979945 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f929cd6a-bbac-41db-9c5f-adfa24bcc5b7 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:01.990952 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 782/1559] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:01.992161 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc0b01-bf74-41ef-9444-2801e065e969 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:01.992445 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc0b01-bf74-41ef-9444-2801e065e969 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:01.992715 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc0b01-bf74-41ef-9444-2801e065e969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:01.999017 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 778/1560] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:26:01 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:01.999844 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49171fa-9ab9-4068-87b2-93eb7d81b2d9 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:02.012629 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-97dc0b01-bf74-41ef-9444-2801e065e969 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:02.013830 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-97dc0b01-bf74-41ef-9444-2801e065e969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:02.061931 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49171fa-9ab9-4068-87b2-93eb7d81b2d9 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:02.064552 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:02.066506 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15133cb5-1e7a-403b-8c69-50bbecdad43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:02.067186 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15133cb5-1e7a-403b-8c69-50bbecdad43f admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:02.067493 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15133cb5-1e7a-403b-8c69-50bbecdad43f admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:02.067769 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15133cb5-1e7a-403b-8c69-50bbecdad43f admin admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:02.068431 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15133cb5-1e7a-403b-8c69-50bbecdad43f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:26:02.068722 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15133cb5-1e7a-403b-8c69-50bbecdad43f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:02.070451 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15133cb5-1e7a-403b-8c69-50bbecdad43f admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:02.074960 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 783/1561] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:02.075918 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dc0b01-bf74-41ef-9444-2801e065e969 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:02.079291 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:02.090539 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 779/1562] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:26:02.090539 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15133cb5-1e7a-403b-8c69-50bbecdad43f admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:02.090836 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dc0b01-bf74-41ef-9444-2801e065e969 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:02.091693 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112d7f25-7ad8-4889-85b9-af0fae63733c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:02.092314 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d7f25-7ad8-4889-85b9-af0fae63733c admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:02.092589 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d7f25-7ad8-4889-85b9-af0fae63733c admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:02.092858 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d7f25-7ad8-4889-85b9-af0fae63733c admin admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:02.093496 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112d7f25-7ad8-4889-85b9-af0fae63733c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:26:02.093791 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112d7f25-7ad8-4889-85b9-af0fae63733c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:02.095857 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112d7f25-7ad8-4889-85b9-af0fae63733c admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:02.099814 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15133cb5-1e7a-403b-8c69-50bbecdad43f admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:02.105274 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e4ef3-59bb-43f0-a106-5463072d4f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:02.105875 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e4ef3-59bb-43f0-a106-5463072d4f67 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:02.106158 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e4ef3-59bb-43f0-a106-5463072d4f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:02.106432 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e4ef3-59bb-43f0-a106-5463072d4f67 admin admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:02.107106 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97e4ef3-59bb-43f0-a106-5463072d4f67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:26:02.107437 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97e4ef3-59bb-43f0-a106-5463072d4f67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:02.108374 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 784/1563] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:26:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:26:02.109167 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97e4ef3-59bb-43f0-a106-5463072d4f67 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:02.109428 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c04f0-ede9-431c-af47-95b76db5949f None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:02.109713 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c04f0-ede9-431c-af47-95b76db5949f None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:02.109980 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c04f0-ede9-431c-af47-95b76db5949f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:02.121918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 780/1564] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:26:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:26:02.122557 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e4ef3-59bb-43f0-a106-5463072d4f67 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:02.128980 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-0b4c04f0-ede9-431c-af47-95b76db5949f None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:02.130266 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-0b4c04f0-ede9-431c-af47-95b76db5949f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:02.132133 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e4ef3-59bb-43f0-a106-5463072d4f67 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:02.136721 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd1be42-1924-4b6a-b9ed-109645258fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:02.137311 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd1be42-1924-4b6a-b9ed-109645258fab admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:02.137588 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd1be42-1924-4b6a-b9ed-109645258fab admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:02.137860 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd1be42-1924-4b6a-b9ed-109645258fab admin admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:02.138385 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 785/1565] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:02.138488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd1be42-1924-4b6a-b9ed-109645258fab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:26:02.139195 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4c04f0-ede9-431c-af47-95b76db5949f None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:02.139459 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd1be42-1924-4b6a-b9ed-109645258fab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:02.141051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd1be42-1924-4b6a-b9ed-109645258fab admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:02.151895 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4c04f0-ede9-431c-af47-95b76db5949f None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:02.161116 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 781/1566] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:26:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:26:02.161116 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd1be42-1924-4b6a-b9ed-109645258fab admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:02.161401 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2a31d-c9b6-472d-ac7f-6c7e9abafb5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:02.161401 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2a31d-c9b6-472d-ac7f-6c7e9abafb5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:02.161401 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2a31d-c9b6-472d-ac7f-6c7e9abafb5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:02.161401 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2a31d-c9b6-472d-ac7f-6c7e9abafb5f admin admin] PATH_INFO: `/v3/projects/1b4039e86e054ac7bcbc85087a73e6fc/users/cfe362eac2a143dd82d2bb0688441567/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:02.165141 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd1be42-1924-4b6a-b9ed-109645258fab admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:02.169719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93fb4f-7167-4cfb-af00-e991a74475b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:02.170305 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93fb4f-7167-4cfb-af00-e991a74475b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:02.170578 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93fb4f-7167-4cfb-af00-e991a74475b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:02.170879 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93fb4f-7167-4cfb-af00-e991a74475b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:02.171504 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93fb4f-7167-4cfb-af00-e991a74475b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:02.172552 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: WARNING py.warnings [None req-cb93fb4f-7167-4cfb-af00-e991a74475b4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 14:26:02.172552 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: warnings.warn(msg) Sep 24 14:26:02.172552 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: Sep 24 14:26:02.173771 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93fb4f-7167-4cfb-af00-e991a74475b4 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:02.174853 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d2a31d-c9b6-472d-ac7f-6c7e9abafb5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b4039e86e054ac7bcbc85087a73e6fc, user_id=cfe362eac2a143dd82d2bb0688441567, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:02.175892 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: WARNING py.warnings [None req-34d2a31d-c9b6-472d-ac7f-6c7e9abafb5f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 14:26:02.175892 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: warnings.warn(msg) Sep 24 14:26:02.175892 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: Sep 24 14:26:02.177861 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d2a31d-c9b6-472d-ac7f-6c7e9abafb5f admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:02.194491 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 786/1567] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:02 2022] PUT /identity/v3/projects/1b4039e86e054ac7bcbc85087a73e6fc/users/cfe362eac2a143dd82d2bb0688441567/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:02.196021 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2a31d-c9b6-472d-ac7f-6c7e9abafb5f admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:02.197866 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 782/1568] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:26:02 2022] POST /identity/v3/users => generated 290 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:02.198548 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93fb4f-7167-4cfb-af00-e991a74475b4 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:02.208632 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93fb4f-7167-4cfb-af00-e991a74475b4 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:02.263559 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00934abc-c2c8-4886-a257-ef0f659f36ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:02.264165 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00934abc-c2c8-4886-a257-ef0f659f36ab admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:02.264440 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00934abc-c2c8-4886-a257-ef0f659f36ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:02.264710 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00934abc-c2c8-4886-a257-ef0f659f36ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:02.265337 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00934abc-c2c8-4886-a257-ef0f659f36ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:02.267261 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00934abc-c2c8-4886-a257-ef0f659f36ab admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:02.270288 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2a31d-c9b6-472d-ac7f-6c7e9abafb5f admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:02.275481 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c5f130-b660-4803-8740-4dd1b11a3965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:02.276087 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c5f130-b660-4803-8740-4dd1b11a3965 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:02.276362 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c5f130-b660-4803-8740-4dd1b11a3965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:02.276991 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c5f130-b660-4803-8740-4dd1b11a3965 admin admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:02.277265 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c5f130-b660-4803-8740-4dd1b11a3965 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:26:02.277562 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c5f130-b660-4803-8740-4dd1b11a3965 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:02.279179 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c5f130-b660-4803-8740-4dd1b11a3965 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:02.292477 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 787/1569] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:26:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:26:02.293273 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c5f130-b660-4803-8740-4dd1b11a3965 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:02.300100 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 783/1570] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:26:02 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:02.300900 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00934abc-c2c8-4886-a257-ef0f659f36ab admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:02.308052 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c5f130-b660-4803-8740-4dd1b11a3965 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:02.362448 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00934abc-c2c8-4886-a257-ef0f659f36ab admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:02.365044 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5d29a-b0ac-46da-af6b-c507617a1ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:02.365651 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5d29a-b0ac-46da-af6b-c507617a1ee5 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:02.365972 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5d29a-b0ac-46da-af6b-c507617a1ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:02.366191 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5d29a-b0ac-46da-af6b-c507617a1ee5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:02.366839 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5d29a-b0ac-46da-af6b-c507617a1ee5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:02.367134 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed39fb8-4004-4086-84a3-44f96493b32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:02.367762 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed39fb8-4004-4086-84a3-44f96493b32a admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:02.368039 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed39fb8-4004-4086-84a3-44f96493b32a admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:02.368308 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed39fb8-4004-4086-84a3-44f96493b32a admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:02.368776 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5d29a-b0ac-46da-af6b-c507617a1ee5 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:02.368926 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed39fb8-4004-4086-84a3-44f96493b32a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:02.370939 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed39fb8-4004-4086-84a3-44f96493b32a admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:02.396107 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 788/1571] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:26:02 2022] POST /identity/v3/projects => generated 405 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:02.399896 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 784/1572] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:26:02 2022] POST /identity/v3/projects => generated 371 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:02.400744 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed39fb8-4004-4086-84a3-44f96493b32a admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:02.403604 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5d29a-b0ac-46da-af6b-c507617a1ee5 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:02.430552 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:02.437491 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:02.464592 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed39fb8-4004-4086-84a3-44f96493b32a admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:02.467033 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5d29a-b0ac-46da-af6b-c507617a1ee5 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:02.469282 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61207e4-2825-4852-a658-a8d80f4a3cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:02.469881 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61207e4-2825-4852-a658-a8d80f4a3cf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:02.470157 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61207e4-2825-4852-a658-a8d80f4a3cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:02.470424 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61207e4-2825-4852-a658-a8d80f4a3cf2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:02.471089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61207e4-2825-4852-a658-a8d80f4a3cf2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:26:02.471387 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61207e4-2825-4852-a658-a8d80f4a3cf2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:02.471585 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b97cca-316b-4425-a52a-db9cf0b5031d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:02.472184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b97cca-316b-4425-a52a-db9cf0b5031d admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:02.472461 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b97cca-316b-4425-a52a-db9cf0b5031d admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:02.472732 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b97cca-316b-4425-a52a-db9cf0b5031d admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:02.472970 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61207e4-2825-4852-a658-a8d80f4a3cf2 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:02.473355 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b97cca-316b-4425-a52a-db9cf0b5031d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:02.475276 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b97cca-316b-4425-a52a-db9cf0b5031d admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:02.484928 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 785/1573] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:26:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:26:02.485707 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61207e4-2825-4852-a658-a8d80f4a3cf2 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:02.496422 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61207e4-2825-4852-a658-a8d80f4a3cf2 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:02.501116 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbdfe8d-5aef-4835-9e58-1c737baa30fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:02.501698 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdfe8d-5aef-4835-9e58-1c737baa30fd admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:02.501973 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdfe8d-5aef-4835-9e58-1c737baa30fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:02.502241 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdfe8d-5aef-4835-9e58-1c737baa30fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:02.506062 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbdfe8d-5aef-4835-9e58-1c737baa30fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:02.506225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 789/1574] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:26:02 2022] POST /identity/v3/projects => generated 389 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:02.506225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71ea9f-574d-49c4-9b36-0c6d98b162c1 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:02.506225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71ea9f-574d-49c4-9b36-0c6d98b162c1 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:02.506516 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbdfe8d-5aef-4835-9e58-1c737baa30fd admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:02.506606 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71ea9f-574d-49c4-9b36-0c6d98b162c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:02.518246 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 786/1575] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:02.518246 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbdfe8d-5aef-4835-9e58-1c737baa30fd admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:02.526100 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbdfe8d-5aef-4835-9e58-1c737baa30fd admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:02.534273 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-8d71ea9f-574d-49c4-9b36-0c6d98b162c1 None None] MFA Rules not processed for user `cfe362eac2a143dd82d2bb0688441567`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:02.535517 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-8d71ea9f-574d-49c4-9b36-0c6d98b162c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:02.583669 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22822a96-5eeb-423f-adfd-2da8cdb09f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:02.584290 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22822a96-5eeb-423f-adfd-2da8cdb09f9c admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:02.584573 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22822a96-5eeb-423f-adfd-2da8cdb09f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:02.584842 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22822a96-5eeb-423f-adfd-2da8cdb09f9c admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:02.585464 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22822a96-5eeb-423f-adfd-2da8cdb09f9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:02.587918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22822a96-5eeb-423f-adfd-2da8cdb09f9c admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:02.602598 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 790/1576] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:02 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:02.602598 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d71ea9f-574d-49c4-9b36-0c6d98b162c1 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:02.610458 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d71ea9f-574d-49c4-9b36-0c6d98b162c1 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:02.615386 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a1152-d768-47ac-97ef-aa8a339533f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:02.615917 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a1152-d768-47ac-97ef-aa8a339533f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:02.616193 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a1152-d768-47ac-97ef-aa8a339533f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:02.616457 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a1152-d768-47ac-97ef-aa8a339533f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:02.617106 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8a1152-d768-47ac-97ef-aa8a339533f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:02.622648 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 787/1577] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:26:02 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:02.622866 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8a1152-d768-47ac-97ef-aa8a339533f0 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:02.622984 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22822a96-5eeb-423f-adfd-2da8cdb09f9c admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:02.641724 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 791/1578] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:26:02 2022] POST /identity/v3/users => generated 288 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:02.642780 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a1152-d768-47ac-97ef-aa8a339533f0 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:02.652075 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a1152-d768-47ac-97ef-aa8a339533f0 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:02.684902 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22822a96-5eeb-423f-adfd-2da8cdb09f9c admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:02.689829 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d757d6-ffc8-425d-993c-b9ce2a115a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:02.690433 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d757d6-ffc8-425d-993c-b9ce2a115a19 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:02.690714 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d757d6-ffc8-425d-993c-b9ce2a115a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:02.691034 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d757d6-ffc8-425d-993c-b9ce2a115a19 admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:02.691662 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d757d6-ffc8-425d-993c-b9ce2a115a19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:02.693583 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d757d6-ffc8-425d-993c-b9ce2a115a19 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:02.710883 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f17f5-91ee-411a-be11-5f1cc46ca568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:02.711519 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f17f5-91ee-411a-be11-5f1cc46ca568 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:02.711797 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f17f5-91ee-411a-be11-5f1cc46ca568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:02.712066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f17f5-91ee-411a-be11-5f1cc46ca568 admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:02.712699 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3f17f5-91ee-411a-be11-5f1cc46ca568 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:02.714666 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3f17f5-91ee-411a-be11-5f1cc46ca568 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:02.715400 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 788/1579] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:26:02 2022] POST /identity/v3/users => generated 303 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:02.718651 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d757d6-ffc8-425d-993c-b9ce2a115a19 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:02.726072 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d757d6-ffc8-425d-993c-b9ce2a115a19 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:02.731250 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c721f-1baa-4617-a796-89c6bb26652f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:02.731963 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c721f-1baa-4617-a796-89c6bb26652f admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:02.732330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c721f-1baa-4617-a796-89c6bb26652f admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:02.732607 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c721f-1baa-4617-a796-89c6bb26652f admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:02.733338 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906c721f-1baa-4617-a796-89c6bb26652f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:02.735328 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906c721f-1baa-4617-a796-89c6bb26652f admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:02.740697 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 792/1580] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:26:02 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:02.741501 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f17f5-91ee-411a-be11-5f1cc46ca568 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:02.751407 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f17f5-91ee-411a-be11-5f1cc46ca568 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:02.756132 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ecd674-4a8b-485d-9a34-031100fec53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:02.756762 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecd674-4a8b-485d-9a34-031100fec53b admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:02.757037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecd674-4a8b-485d-9a34-031100fec53b admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:02.757301 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecd674-4a8b-485d-9a34-031100fec53b admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:02.758600 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ecd674-4a8b-485d-9a34-031100fec53b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:02.760705 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ecd674-4a8b-485d-9a34-031100fec53b admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:02.773226 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 789/1581] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:26:02 2022] POST /identity/v3/projects => generated 393 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:02.774270 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c721f-1baa-4617-a796-89c6bb26652f admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:02.779046 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:02.783129 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 793/1582] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:26:02 2022] POST /identity/v3/users => generated 295 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:02.784272 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ecd674-4a8b-485d-9a34-031100fec53b admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:02.791398 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:02.797062 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ecd674-4a8b-485d-9a34-031100fec53b admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:02.839583 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c721f-1baa-4617-a796-89c6bb26652f admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:02.844968 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01018ff5-8d1e-43ee-b754-69efdbaf29f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:02.845675 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01018ff5-8d1e-43ee-b754-69efdbaf29f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:02.846004 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01018ff5-8d1e-43ee-b754-69efdbaf29f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:02.846280 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01018ff5-8d1e-43ee-b754-69efdbaf29f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:02.846949 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01018ff5-8d1e-43ee-b754-69efdbaf29f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:02.848543 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01018ff5-8d1e-43ee-b754-69efdbaf29f8 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:02.857084 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d3f0a-a8bb-4f79-836f-5dbeb348fc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:02.857692 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d3f0a-a8bb-4f79-836f-5dbeb348fc7f admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:02.858021 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d3f0a-a8bb-4f79-836f-5dbeb348fc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:02.858318 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d3f0a-a8bb-4f79-836f-5dbeb348fc7f admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:02.858418 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 790/1583] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:02.858974 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603d3f0a-a8bb-4f79-836f-5dbeb348fc7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:02.859169 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01018ff5-8d1e-43ee-b754-69efdbaf29f8 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:02.860927 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603d3f0a-a8bb-4f79-836f-5dbeb348fc7f admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:02.870952 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01018ff5-8d1e-43ee-b754-69efdbaf29f8 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:02.875523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9168a993-b0d6-4405-acae-87ebeb17dc3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:02.876109 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168a993-b0d6-4405-acae-87ebeb17dc3c admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:02.876416 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168a993-b0d6-4405-acae-87ebeb17dc3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:02.876685 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168a993-b0d6-4405-acae-87ebeb17dc3c admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:02.877307 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9168a993-b0d6-4405-acae-87ebeb17dc3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:02.879300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9168a993-b0d6-4405-acae-87ebeb17dc3c admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:02.889200 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 794/1584] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:26:02 2022] POST /identity/v3/projects => generated 355 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:02.889846 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d3f0a-a8bb-4f79-836f-5dbeb348fc7f admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:02.907961 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 791/1585] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:26:02 2022] POST /identity/v3/users => generated 287 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:02.907961 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9168a993-b0d6-4405-acae-87ebeb17dc3c admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:02.916036 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9168a993-b0d6-4405-acae-87ebeb17dc3c admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:02.954089 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d3f0a-a8bb-4f79-836f-5dbeb348fc7f admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:02.958725 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22070afc-d9af-44cd-a2f9-ca518cb5d7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:02.959360 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22070afc-d9af-44cd-a2f9-ca518cb5d7e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:02.959668 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22070afc-d9af-44cd-a2f9-ca518cb5d7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:02.959938 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22070afc-d9af-44cd-a2f9-ca518cb5d7e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:02.960579 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22070afc-d9af-44cd-a2f9-ca518cb5d7e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:02.962211 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22070afc-d9af-44cd-a2f9-ca518cb5d7e4 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:02.972612 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 795/1586] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:02.972769 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a24249-5c2f-4904-8189-4b61bb00113c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:02.973247 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a24249-5c2f-4904-8189-4b61bb00113c admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:02.973501 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22070afc-d9af-44cd-a2f9-ca518cb5d7e4 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:02.973621 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a24249-5c2f-4904-8189-4b61bb00113c admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:02.973780 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a24249-5c2f-4904-8189-4b61bb00113c admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:02.974408 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a24249-5c2f-4904-8189-4b61bb00113c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:02.976011 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a24249-5c2f-4904-8189-4b61bb00113c admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:02.985309 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22070afc-d9af-44cd-a2f9-ca518cb5d7e4 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:02.986428 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 792/1587] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:02.990268 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a24249-5c2f-4904-8189-4b61bb00113c admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:02.990622 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570df0c-bf8a-4096-9926-5cb75cf86035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:02.991248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570df0c-bf8a-4096-9926-5cb75cf86035 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:02.991525 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570df0c-bf8a-4096-9926-5cb75cf86035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:02.991791 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570df0c-bf8a-4096-9926-5cb75cf86035 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:02.992971 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7570df0c-bf8a-4096-9926-5cb75cf86035 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:02.994538 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7570df0c-bf8a-4096-9926-5cb75cf86035 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:03.001442 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a24249-5c2f-4904-8189-4b61bb00113c admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:03.006278 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 796/1588] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:03.006538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65aaccd-f33d-40f8-9b8d-02a54c206c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:03.007102 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65aaccd-f33d-40f8-9b8d-02a54c206c95 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:03.007328 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65aaccd-f33d-40f8-9b8d-02a54c206c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:03.007601 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65aaccd-f33d-40f8-9b8d-02a54c206c95 admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:03.008268 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65aaccd-f33d-40f8-9b8d-02a54c206c95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:03.010183 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65aaccd-f33d-40f8-9b8d-02a54c206c95 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:03.010343 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570df0c-bf8a-4096-9926-5cb75cf86035 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:03.020394 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570df0c-bf8a-4096-9926-5cb75cf86035 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:03.025399 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce137553-b60d-403a-8cd4-482b67092a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:03.026002 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce137553-b60d-403a-8cd4-482b67092a24 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:03.026293 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce137553-b60d-403a-8cd4-482b67092a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:03.026606 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce137553-b60d-403a-8cd4-482b67092a24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:03.027272 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce137553-b60d-403a-8cd4-482b67092a24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:03.028861 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce137553-b60d-403a-8cd4-482b67092a24 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:03.042301 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 797/1589] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:03.043163 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce137553-b60d-403a-8cd4-482b67092a24 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:03.050602 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 793/1590] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:26:02 2022] POST /identity/v3/users => generated 297 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:03.051425 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65aaccd-f33d-40f8-9b8d-02a54c206c95 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:03.052911 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce137553-b60d-403a-8cd4-482b67092a24 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:03.058035 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f5d034-6820-41f5-bdc2-a26c32926deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:03.058678 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5d034-6820-41f5-bdc2-a26c32926deb admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:03.058984 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5d034-6820-41f5-bdc2-a26c32926deb admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:03.059256 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5d034-6820-41f5-bdc2-a26c32926deb admin admin] PATH_INFO: `/v3/projects/6bb2f863f5834cd4a47a3ed9feda7c02/users/75575c10a8eb4ebfab5b2f0c330541df/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:03.062240 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65aaccd-f33d-40f8-9b8d-02a54c206c95 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:03.066746 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad027e-d786-4ca9-ad29-fcbe92a91d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:03.067370 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad027e-d786-4ca9-ad29-fcbe92a91d8a admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:03.067642 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad027e-d786-4ca9-ad29-fcbe92a91d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:03.067913 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad027e-d786-4ca9-ad29-fcbe92a91d8a admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:03.068538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ad027e-d786-4ca9-ad29-fcbe92a91d8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:03.070481 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ad027e-d786-4ca9-ad29-fcbe92a91d8a admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:03.074509 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f5d034-6820-41f5-bdc2-a26c32926deb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bb2f863f5834cd4a47a3ed9feda7c02, user_id=75575c10a8eb4ebfab5b2f0c330541df, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:03.077393 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f5d034-6820-41f5-bdc2-a26c32926deb admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:03.091500 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 798/1591] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:03 2022] PUT /identity/v3/projects/6bb2f863f5834cd4a47a3ed9feda7c02/users/75575c10a8eb4ebfab5b2f0c330541df/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:03.092434 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f5d034-6820-41f5-bdc2-a26c32926deb admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:03.098708 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 794/1592] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:26:03 2022] POST /identity/v3/users => generated 284 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:03.098708 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad027e-d786-4ca9-ad29-fcbe92a91d8a admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:03.106751 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad027e-d786-4ca9-ad29-fcbe92a91d8a admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:03.137981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:03.147860 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:03.160235 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f5d034-6820-41f5-bdc2-a26c32926deb admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:03.165366 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eb0460-9ad3-464c-bd3d-e784bc6e071c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:03.165966 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb0460-9ad3-464c-bd3d-e784bc6e071c admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:03.166180 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14538a9-2699-4a37-a509-01128015b351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:03.166326 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb0460-9ad3-464c-bd3d-e784bc6e071c admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:03.166491 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb0460-9ad3-464c-bd3d-e784bc6e071c admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:03.166790 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14538a9-2699-4a37-a509-01128015b351 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:03.167112 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14538a9-2699-4a37-a509-01128015b351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:03.167392 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14538a9-2699-4a37-a509-01128015b351 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:03.167582 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eb0460-9ad3-464c-bd3d-e784bc6e071c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:03.168033 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14538a9-2699-4a37-a509-01128015b351 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:03.169172 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eb0460-9ad3-464c-bd3d-e784bc6e071c admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:03.169620 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14538a9-2699-4a37-a509-01128015b351 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:03.179394 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 799/1593] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:03.179696 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 795/1594] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:03.181107 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14538a9-2699-4a37-a509-01128015b351 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:03.186579 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eb0460-9ad3-464c-bd3d-e784bc6e071c admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:03.190750 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14538a9-2699-4a37-a509-01128015b351 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:03.195292 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8b39a-951f-46fc-9e9c-746eaa465a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:03.195942 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8b39a-951f-46fc-9e9c-746eaa465a9b admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:03.196049 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eb0460-9ad3-464c-bd3d-e784bc6e071c admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:03.196220 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8b39a-951f-46fc-9e9c-746eaa465a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:03.196490 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8b39a-951f-46fc-9e9c-746eaa465a9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:03.197117 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea8b39a-951f-46fc-9e9c-746eaa465a9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:03.198672 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea8b39a-951f-46fc-9e9c-746eaa465a9b admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:03.200594 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51921ce5-1306-4c6e-9590-8cc7192d993c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:03.201177 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51921ce5-1306-4c6e-9590-8cc7192d993c admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:03.201458 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51921ce5-1306-4c6e-9590-8cc7192d993c admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:03.201728 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51921ce5-1306-4c6e-9590-8cc7192d993c admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:03.202362 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51921ce5-1306-4c6e-9590-8cc7192d993c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:03.203967 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51921ce5-1306-4c6e-9590-8cc7192d993c admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:03.208742 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 796/1595] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:03.209531 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8b39a-951f-46fc-9e9c-746eaa465a9b admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:03.214147 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 800/1596] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:03.214889 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51921ce5-1306-4c6e-9590-8cc7192d993c admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:03.219456 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8b39a-951f-46fc-9e9c-746eaa465a9b admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:03.224029 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23bb294-e55c-4cd4-8fa7-af7c5afbac3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:03.224483 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51921ce5-1306-4c6e-9590-8cc7192d993c admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:03.224712 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23bb294-e55c-4cd4-8fa7-af7c5afbac3a admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:03.224992 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23bb294-e55c-4cd4-8fa7-af7c5afbac3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:03.225262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23bb294-e55c-4cd4-8fa7-af7c5afbac3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:03.225885 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23bb294-e55c-4cd4-8fa7-af7c5afbac3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:03.227971 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23bb294-e55c-4cd4-8fa7-af7c5afbac3a admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:03.229488 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5abdb7-e156-4c3c-bfec-731ab3d3d67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:03.230078 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5abdb7-e156-4c3c-bfec-731ab3d3d67f admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:03.230333 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5abdb7-e156-4c3c-bfec-731ab3d3d67f admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:03.230602 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5abdb7-e156-4c3c-bfec-731ab3d3d67f admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:03.231259 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5abdb7-e156-4c3c-bfec-731ab3d3d67f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:03.232802 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5abdb7-e156-4c3c-bfec-731ab3d3d67f admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:03.238214 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 797/1597] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:03.239381 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85f64b6-f702-4f5f-afbf-429274e57051 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:03.239660 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85f64b6-f702-4f5f-afbf-429274e57051 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:03.239926 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85f64b6-f702-4f5f-afbf-429274e57051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:03.243837 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 801/1598] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:03.247613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5abdb7-e156-4c3c-bfec-731ab3d3d67f admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:03.258006 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5abdb7-e156-4c3c-bfec-731ab3d3d67f admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:03.259080 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-f85f64b6-f702-4f5f-afbf-429274e57051 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:03.260277 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-f85f64b6-f702-4f5f-afbf-429274e57051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:03.262572 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c81a8-3ca9-428b-83c9-4858d8877159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:03.263180 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c81a8-3ca9-428b-83c9-4858d8877159 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:03.263457 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c81a8-3ca9-428b-83c9-4858d8877159 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:03.263734 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c81a8-3ca9-428b-83c9-4858d8877159 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:03.264377 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c81a8-3ca9-428b-83c9-4858d8877159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:03.266001 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c81a8-3ca9-428b-83c9-4858d8877159 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:03.268150 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 798/1599] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:03.268783 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85f64b6-f702-4f5f-afbf-429274e57051 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:03.275857 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 802/1600] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:03.276640 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c81a8-3ca9-428b-83c9-4858d8877159 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:03.281911 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85f64b6-f702-4f5f-afbf-429274e57051 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:03.286791 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b443844a-a644-4a00-ac0e-ac603040ef35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:03.287413 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b443844a-a644-4a00-ac0e-ac603040ef35 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:03.287687 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b443844a-a644-4a00-ac0e-ac603040ef35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:03.287953 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b443844a-a644-4a00-ac0e-ac603040ef35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:03.288273 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c81a8-3ca9-428b-83c9-4858d8877159 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:03.288582 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b443844a-a644-4a00-ac0e-ac603040ef35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:03.290658 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b443844a-a644-4a00-ac0e-ac603040ef35 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:03.293459 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336b5a4-d10e-4c37-9136-84d4171eda64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:03.294089 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336b5a4-d10e-4c37-9136-84d4171eda64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:03.294366 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336b5a4-d10e-4c37-9136-84d4171eda64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:03.294636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336b5a4-d10e-4c37-9136-84d4171eda64 admin admin] PATH_INFO: `/v3/projects/d05d72126e6d4ec58cee3cc804a93fb6/users/2038b90f689442489fb00fc6761d28c0/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:03.300590 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 799/1601] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:03.301371 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b443844a-a644-4a00-ac0e-ac603040ef35 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:03.310241 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d336b5a4-d10e-4c37-9136-84d4171eda64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d05d72126e6d4ec58cee3cc804a93fb6, user_id=2038b90f689442489fb00fc6761d28c0, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:03.311283 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b443844a-a644-4a00-ac0e-ac603040ef35 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:03.313003 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d336b5a4-d10e-4c37-9136-84d4171eda64 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:03.315818 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df2a38f-e3c4-40b4-9c2a-9c6fb13d28be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:03.316453 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2a38f-e3c4-40b4-9c2a-9c6fb13d28be admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:03.316737 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2a38f-e3c4-40b4-9c2a-9c6fb13d28be admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:03.317014 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2a38f-e3c4-40b4-9c2a-9c6fb13d28be admin admin] PATH_INFO: `/v3/projects/18d5cf0251234c518d493e8a529999d0/users/9c0f213d9cbb4418a759559972ab748d/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:03.326286 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 803/1602] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:03 2022] PUT /identity/v3/projects/d05d72126e6d4ec58cee3cc804a93fb6/users/2038b90f689442489fb00fc6761d28c0/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:03.327159 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336b5a4-d10e-4c37-9136-84d4171eda64 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:03.335047 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df2a38f-e3c4-40b4-9c2a-9c6fb13d28be admin admin] RBAC: Authorizing `identity:create_grant(project_id=18d5cf0251234c518d493e8a529999d0, user_id=9c0f213d9cbb4418a759559972ab748d, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:03.338349 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df2a38f-e3c4-40b4-9c2a-9c6fb13d28be admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:03.354885 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 800/1603] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:03 2022] PUT /identity/v3/projects/18d5cf0251234c518d493e8a529999d0/users/9c0f213d9cbb4418a759559972ab748d/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:03.355664 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df2a38f-e3c4-40b4-9c2a-9c6fb13d28be admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:03.422389 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df2a38f-e3c4-40b4-9c2a-9c6fb13d28be admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:03.426305 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336b5a4-d10e-4c37-9136-84d4171eda64 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:03.426908 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f288cf-d9f0-4859-84de-7f4a7c49a42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:03.427564 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f288cf-d9f0-4859-84de-7f4a7c49a42f admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:03.427841 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f288cf-d9f0-4859-84de-7f4a7c49a42f admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:03.428114 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f288cf-d9f0-4859-84de-7f4a7c49a42f admin admin] PATH_INFO: `/v3/projects/1f7134ad65014cb984417c99b3d76eff/users/883e6570afb84a099e46cdace5a88b2e/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:03.431976 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c7eb56-fa5b-43cc-a8e5-d8b9f1bd646b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:03.432627 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7eb56-fa5b-43cc-a8e5-d8b9f1bd646b admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:03.432905 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7eb56-fa5b-43cc-a8e5-d8b9f1bd646b admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:03.433175 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7eb56-fa5b-43cc-a8e5-d8b9f1bd646b admin admin] PATH_INFO: `/v3/projects/e25e868cd2d44dfe964975ce9268b576/users/065701e193524356bf2846785eebf68e/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:03.443734 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f288cf-d9f0-4859-84de-7f4a7c49a42f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f7134ad65014cb984417c99b3d76eff, user_id=883e6570afb84a099e46cdace5a88b2e, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:03.446525 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f288cf-d9f0-4859-84de-7f4a7c49a42f admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:03.448171 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c7eb56-fa5b-43cc-a8e5-d8b9f1bd646b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e25e868cd2d44dfe964975ce9268b576, user_id=065701e193524356bf2846785eebf68e, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:03.450835 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c7eb56-fa5b-43cc-a8e5-d8b9f1bd646b admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:03.461805 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 801/1604] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:03 2022] PUT /identity/v3/projects/1f7134ad65014cb984417c99b3d76eff/users/883e6570afb84a099e46cdace5a88b2e/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:03.465662 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f288cf-d9f0-4859-84de-7f4a7c49a42f admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:03.465778 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 804/1605] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:03 2022] PUT /identity/v3/projects/e25e868cd2d44dfe964975ce9268b576/users/065701e193524356bf2846785eebf68e/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:03.466151 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c7eb56-fa5b-43cc-a8e5-d8b9f1bd646b admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:03.523467 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c7eb56-fa5b-43cc-a8e5-d8b9f1bd646b admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:03.527851 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c409555a-92b6-48ff-8520-eb2a032d1c00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:03.528459 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c409555a-92b6-48ff-8520-eb2a032d1c00 service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:03.528732 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c409555a-92b6-48ff-8520-eb2a032d1c00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:03.529008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c409555a-92b6-48ff-8520-eb2a032d1c00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:03.529401 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:03.533363 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f288cf-d9f0-4859-84de-7f4a7c49a42f admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:03.542776 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:03.593115 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e715f7a-0acb-4111-b3f7-a188ea89198f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:03.593332 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c409555a-92b6-48ff-8520-eb2a032d1c00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:03.593721 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e715f7a-0acb-4111-b3f7-a188ea89198f admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:03.594000 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e715f7a-0acb-4111-b3f7-a188ea89198f admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:03.594296 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e715f7a-0acb-4111-b3f7-a188ea89198f admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:03.594683 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c409555a-92b6-48ff-8520-eb2a032d1c00 service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:03.594969 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e715f7a-0acb-4111-b3f7-a188ea89198f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:03.596897 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e715f7a-0acb-4111-b3f7-a188ea89198f admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:03.604872 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 805/1606] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:03.605473 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c409555a-92b6-48ff-8520-eb2a032d1c00 service neutron] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:03.610449 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 802/1607] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:03.610449 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e715f7a-0acb-4111-b3f7-a188ea89198f admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:03.615197 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c409555a-92b6-48ff-8520-eb2a032d1c00 service neutron] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:03.620887 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0d3c00-f651-4da8-b138-70280278d0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:03.621296 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e715f7a-0acb-4111-b3f7-a188ea89198f admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:03.621539 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d3c00-f651-4da8-b138-70280278d0bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:03.621843 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d3c00-f651-4da8-b138-70280278d0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:03.622116 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d3c00-f651-4da8-b138-70280278d0bc admin admin] PATH_INFO: `/v3/projects/0035103def854f6b8286a7765b430525/users/47bc881aa5e84ead9cd551c0fdd13d15/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:03.627381 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9fa74-a9a8-41c2-a56c-0e946e93317b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:03.628326 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9fa74-a9a8-41c2-a56c-0e946e93317b admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:03.628677 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9fa74-a9a8-41c2-a56c-0e946e93317b admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:03.628951 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9fa74-a9a8-41c2-a56c-0e946e93317b admin admin] PATH_INFO: `/v3/projects/c8d85d19f92d43d38b05bf9a5a73f550/users/2d15654b29bd450f9af8e39b9119e3fa/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:03.638374 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0d3c00-f651-4da8-b138-70280278d0bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0035103def854f6b8286a7765b430525, user_id=47bc881aa5e84ead9cd551c0fdd13d15, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:03.640890 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0d3c00-f651-4da8-b138-70280278d0bc admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:03.646630 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b9fa74-a9a8-41c2-a56c-0e946e93317b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8d85d19f92d43d38b05bf9a5a73f550, user_id=2d15654b29bd450f9af8e39b9119e3fa, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:03.650218 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b9fa74-a9a8-41c2-a56c-0e946e93317b admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:03.656276 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 806/1608] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:03 2022] PUT /identity/v3/projects/0035103def854f6b8286a7765b430525/users/47bc881aa5e84ead9cd551c0fdd13d15/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:03.659583 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23699283-42ff-4920-a687-181b2233764d None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:03.659583 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23699283-42ff-4920-a687-181b2233764d None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:03.659583 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23699283-42ff-4920-a687-181b2233764d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:03.669063 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 803/1609] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:03 2022] PUT /identity/v3/projects/c8d85d19f92d43d38b05bf9a5a73f550/users/2d15654b29bd450f9af8e39b9119e3fa/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:03.670539 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7563f84-f4ef-4707-9e05-2aa6220a2954 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:03.671060 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7563f84-f4ef-4707-9e05-2aa6220a2954 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:03.671476 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7563f84-f4ef-4707-9e05-2aa6220a2954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:03.689661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-23699283-42ff-4920-a687-181b2233764d None None] MFA Rules not processed for user `2038b90f689442489fb00fc6761d28c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:03.690904 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-23699283-42ff-4920-a687-181b2233764d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:03.707970 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-a7563f84-f4ef-4707-9e05-2aa6220a2954 None None] MFA Rules not processed for user `9c0f213d9cbb4418a759559972ab748d`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:03.709836 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-a7563f84-f4ef-4707-9e05-2aa6220a2954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:03.766217 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 807/1610] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:03.779068 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f2d36-541b-4a10-8b9d-bf72177a55aa None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:03.779354 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f2d36-541b-4a10-8b9d-bf72177a55aa None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:03.779627 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f2d36-541b-4a10-8b9d-bf72177a55aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:03.787011 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 804/1611] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:03.788467 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e4b6a-40e8-4cc8-9ba2-d1ece339265c None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:03.788762 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e4b6a-40e8-4cc8-9ba2-d1ece339265c None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:03.789037 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e4b6a-40e8-4cc8-9ba2-d1ece339265c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:03.828102 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-a84f2d36-541b-4a10-8b9d-bf72177a55aa None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:03.831525 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-a84f2d36-541b-4a10-8b9d-bf72177a55aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:03.832723 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-442e4b6a-40e8-4cc8-9ba2-d1ece339265c None None] MFA Rules not processed for user `065701e193524356bf2846785eebf68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:03.833852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-442e4b6a-40e8-4cc8-9ba2-d1ece339265c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:03.906084 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 808/1612] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:03.907037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f2d36-541b-4a10-8b9d-bf72177a55aa None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:03.915511 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:03.919151 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f2d36-541b-4a10-8b9d-bf72177a55aa None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:03.925733 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c186fdd3-b765-4b51-8ca1-88e858f6421b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:03.926705 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186fdd3-b765-4b51-8ca1-88e858f6421b admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:03.926979 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186fdd3-b765-4b51-8ca1-88e858f6421b admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:03.927187 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186fdd3-b765-4b51-8ca1-88e858f6421b admin admin] PATH_INFO: `/v3/projects/9b458789f89a441caad76e78167bc00b/users/c3cf8e31062d4bd7bfa76c09e7e2adea/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:03.929823 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 805/1613] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:03 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:03.936038 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e4b6a-40e8-4cc8-9ba2-d1ece339265c None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:03.947728 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:03.948966 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e4b6a-40e8-4cc8-9ba2-d1ece339265c None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:03.951927 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c186fdd3-b765-4b51-8ca1-88e858f6421b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b458789f89a441caad76e78167bc00b, user_id=c3cf8e31062d4bd7bfa76c09e7e2adea, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:03.954922 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c186fdd3-b765-4b51-8ca1-88e858f6421b admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:03.955196 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0622ef65-a5f7-4456-8062-2e3d9a0eacc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:03.956190 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0622ef65-a5f7-4456-8062-2e3d9a0eacc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:03.956751 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0622ef65-a5f7-4456-8062-2e3d9a0eacc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:03.957009 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0622ef65-a5f7-4456-8062-2e3d9a0eacc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:03.957872 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0622ef65-a5f7-4456-8062-2e3d9a0eacc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:03.959680 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0622ef65-a5f7-4456-8062-2e3d9a0eacc1 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:03.971822 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 806/1614] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:03.973687 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac726174-b355-4e6d-a603-849f252f5b91 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:03.974093 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac726174-b355-4e6d-a603-849f252f5b91 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:03.974422 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac726174-b355-4e6d-a603-849f252f5b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:03.995671 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 809/1615] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:03 2022] PUT /identity/v3/projects/9b458789f89a441caad76e78167bc00b/users/c3cf8e31062d4bd7bfa76c09e7e2adea/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:03.996511 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c186fdd3-b765-4b51-8ca1-88e858f6421b admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:04.013046 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-ac726174-b355-4e6d-a603-849f252f5b91 None None] MFA Rules not processed for user `2d15654b29bd450f9af8e39b9119e3fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:04.014619 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-ac726174-b355-4e6d-a603-849f252f5b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:04.074926 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c186fdd3-b765-4b51-8ca1-88e858f6421b admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:04.079579 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600b191-5cb2-4ee5-8250-3b99ef1c50cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:04.080220 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600b191-5cb2-4ee5-8250-3b99ef1c50cd admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:04.080487 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600b191-5cb2-4ee5-8250-3b99ef1c50cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:04.080731 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600b191-5cb2-4ee5-8250-3b99ef1c50cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:04.081380 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c600b191-5cb2-4ee5-8250-3b99ef1c50cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:04.083397 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c600b191-5cb2-4ee5-8250-3b99ef1c50cd admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:04.099381 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 807/1616] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:04.101161 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e46032-9063-40ae-aece-e3e08a3547fd None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:04.101161 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e46032-9063-40ae-aece-e3e08a3547fd None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:04.101364 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e46032-9063-40ae-aece-e3e08a3547fd None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:04.102945 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 808/1617] 10.222.0.52 () {54 vars in 961 bytes} [Sat Sep 24 14:26:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:26:04.103911 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e46032-9063-40ae-aece-e3e08a3547fd None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:04.130762 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 810/1618] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:26:03 2022] POST /identity/v3/projects => generated 373 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:04.132018 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600b191-5cb2-4ee5-8250-3b99ef1c50cd admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:04.184637 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e46032-9063-40ae-aece-e3e08a3547fd None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:04.212960 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600b191-5cb2-4ee5-8250-3b99ef1c50cd admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:04.218520 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6768423-7164-4b9d-afeb-4febaf4380eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:04.219270 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6768423-7164-4b9d-afeb-4febaf4380eb admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:04.219552 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6768423-7164-4b9d-afeb-4febaf4380eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:04.220576 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6768423-7164-4b9d-afeb-4febaf4380eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:04.220576 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6768423-7164-4b9d-afeb-4febaf4380eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:04.222474 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6768423-7164-4b9d-afeb-4febaf4380eb admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:04.238443 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b99611e-b6d0-44f2-9038-fbb04ab0e2ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:04.239151 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b99611e-b6d0-44f2-9038-fbb04ab0e2ea service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:04.239434 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b99611e-b6d0-44f2-9038-fbb04ab0e2ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:04.239710 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b99611e-b6d0-44f2-9038-fbb04ab0e2ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:04.252062 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b99611e-b6d0-44f2-9038-fbb04ab0e2ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:04.253530 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b99611e-b6d0-44f2-9038-fbb04ab0e2ea service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:04.260457 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 811/1619] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:26:04 2022] POST /identity/v3/projects => generated 405 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:04.260457 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6768423-7164-4b9d-afeb-4febaf4380eb admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:04.264242 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 809/1620] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:04.265369 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f887c-87cf-416f-a663-aece17bbcdb0 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:04.265650 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f887c-87cf-416f-a663-aece17bbcdb0 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:04.266386 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f887c-87cf-416f-a663-aece17bbcdb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:04.286182 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-150f887c-87cf-416f-a663-aece17bbcdb0 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:04.287435 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-150f887c-87cf-416f-a663-aece17bbcdb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:04.325383 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:04.365125 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 810/1621] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:04.367849 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6768423-7164-4b9d-afeb-4febaf4380eb admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:04.373876 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24091c57-0ee9-4c68-8ec8-eed542db5f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:04.374960 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24091c57-0ee9-4c68-8ec8-eed542db5f54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:04.375473 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a293d-6a83-4cf1-8adb-9da7ac8ca6d7 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:04.375721 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24091c57-0ee9-4c68-8ec8-eed542db5f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:04.375839 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a293d-6a83-4cf1-8adb-9da7ac8ca6d7 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:04.376371 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24091c57-0ee9-4c68-8ec8-eed542db5f54 admin admin] PATH_INFO: `/v3/projects/0035103def854f6b8286a7765b430525/users/47bc881aa5e84ead9cd551c0fdd13d15/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:04.376619 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a293d-6a83-4cf1-8adb-9da7ac8ca6d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:04.378900 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24091c57-0ee9-4c68-8ec8-eed542db5f54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0035103def854f6b8286a7765b430525, user_id=47bc881aa5e84ead9cd551c0fdd13d15, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:04.382182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24091c57-0ee9-4c68-8ec8-eed542db5f54 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:04.387242 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:04.397227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 812/1622] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:04 2022] PUT /identity/v3/projects/0035103def854f6b8286a7765b430525/users/47bc881aa5e84ead9cd551c0fdd13d15/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:04.404887 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24091c57-0ee9-4c68-8ec8-eed542db5f54 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:04.438915 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-717a293d-6a83-4cf1-8adb-9da7ac8ca6d7 None None] MFA Rules not processed for user `79eaaae97aac4282a613939e5b768376`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:04.440432 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-717a293d-6a83-4cf1-8adb-9da7ac8ca6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:04.505829 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24091c57-0ee9-4c68-8ec8-eed542db5f54 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:04.515593 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b9440-a1c7-4917-bcc0-d49910dd9db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:04.516479 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b9440-a1c7-4917-bcc0-d49910dd9db9 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:04.516764 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b9440-a1c7-4917-bcc0-d49910dd9db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:04.517037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b9440-a1c7-4917-bcc0-d49910dd9db9 admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:04.518097 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b9440-a1c7-4917-bcc0-d49910dd9db9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:04.520593 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b9440-a1c7-4917-bcc0-d49910dd9db9 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:04.537704 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 811/1623] 10.222.0.52 () {58 vars in 1058 bytes} [Sat Sep 24 14:26:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:04.542200 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b733c8fa-44b2-48a8-99ad-eb826f1a8b5c None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:04.542736 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b733c8fa-44b2-48a8-99ad-eb826f1a8b5c None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:04.545330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b733c8fa-44b2-48a8-99ad-eb826f1a8b5c None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:04.551297 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 812/1624] 10.222.0.52 () {54 vars in 961 bytes} [Sat Sep 24 14:26:04 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:26:04.552054 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b733c8fa-44b2-48a8-99ad-eb826f1a8b5c None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:04.556753 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 813/1625] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:26:04 2022] POST /identity/v3/users => generated 293 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:04.558004 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b9440-a1c7-4917-bcc0-d49910dd9db9 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:04.572605 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b733c8fa-44b2-48a8-99ad-eb826f1a8b5c None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:04.581069 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830b976-efa1-43b3-a588-050a207b5780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:04.581689 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830b976-efa1-43b3-a588-050a207b5780 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:04.581972 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830b976-efa1-43b3-a588-050a207b5780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:04.582377 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830b976-efa1-43b3-a588-050a207b5780 admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:04.583319 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4830b976-efa1-43b3-a588-050a207b5780 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:04.585704 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4830b976-efa1-43b3-a588-050a207b5780 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:04.612235 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 813/1626] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:26:04 2022] POST /identity/v3/users => generated 309 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:04.615074 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf09bc-fc9f-4a5e-ac70-638a4ea941ff None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:04.615524 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf09bc-fc9f-4a5e-ac70-638a4ea941ff None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:04.615809 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf09bc-fc9f-4a5e-ac70-638a4ea941ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:04.649262 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b9440-a1c7-4917-bcc0-d49910dd9db9 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:04.653741 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dca3ac-d932-4655-b990-26aab80123b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:04.654380 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dca3ac-d932-4655-b990-26aab80123b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:04.654780 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dca3ac-d932-4655-b990-26aab80123b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:04.654998 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dca3ac-d932-4655-b990-26aab80123b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:04.657106 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-1ccf09bc-fc9f-4a5e-ac70-638a4ea941ff None None] MFA Rules not processed for user `47bc881aa5e84ead9cd551c0fdd13d15`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:04.658386 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-1ccf09bc-fc9f-4a5e-ac70-638a4ea941ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:04.667294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dca3ac-d932-4655-b990-26aab80123b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:04.669274 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dca3ac-d932-4655-b990-26aab80123b0 service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:04.682552 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 814/1627] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:04.683684 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dca3ac-d932-4655-b990-26aab80123b0 service neutron] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:04.700916 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dca3ac-d932-4655-b990-26aab80123b0 service neutron] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:04.709945 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac734ef9-cd11-4f90-8a65-05543f16ee8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:04.710668 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac734ef9-cd11-4f90-8a65-05543f16ee8e service nova] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:04.711099 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac734ef9-cd11-4f90-8a65-05543f16ee8e service nova] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:04.711433 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac734ef9-cd11-4f90-8a65-05543f16ee8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:04.810287 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 814/1628] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:04 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:04.812084 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0737303-6bf4-419f-a5de-a6190fc71b7c None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:04.813242 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0737303-6bf4-419f-a5de-a6190fc71b7c None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:04.813242 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0737303-6bf4-419f-a5de-a6190fc71b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:04.843041 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-d0737303-6bf4-419f-a5de-a6190fc71b7c None None] MFA Rules not processed for user `79eaaae97aac4282a613939e5b768376`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:04.845106 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-d0737303-6bf4-419f-a5de-a6190fc71b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:04.846793 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac734ef9-cd11-4f90-8a65-05543f16ee8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:04.850599 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac734ef9-cd11-4f90-8a65-05543f16ee8e service nova] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:04.859840 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 815/1629] 10.222.0.52 () {58 vars in 1058 bytes} [Sat Sep 24 14:26:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:04.862474 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0737303-6bf4-419f-a5de-a6190fc71b7c None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:04.875017 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 815/1630] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:04.875408 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0737303-6bf4-419f-a5de-a6190fc71b7c None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:04.878894 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac734ef9-cd11-4f90-8a65-05543f16ee8e service nova] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:04.883772 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70349299-291c-4667-bd83-3a21f9ae0146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:04.884395 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70349299-291c-4667-bd83-3a21f9ae0146 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:04.885927 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70349299-291c-4667-bd83-3a21f9ae0146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:04.886983 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70349299-291c-4667-bd83-3a21f9ae0146 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:04.889493 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70349299-291c-4667-bd83-3a21f9ae0146 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:04.892477 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac734ef9-cd11-4f90-8a65-05543f16ee8e service nova] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:04.895009 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70349299-291c-4667-bd83-3a21f9ae0146 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:04.901025 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adca9dd-ba35-4f14-8854-a18487b65ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:04.901933 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adca9dd-ba35-4f14-8854-a18487b65ee8 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:04.902441 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adca9dd-ba35-4f14-8854-a18487b65ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:04.902979 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adca9dd-ba35-4f14-8854-a18487b65ee8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:04.903849 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adca9dd-ba35-4f14-8854-a18487b65ee8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:04.906977 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adca9dd-ba35-4f14-8854-a18487b65ee8 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:04.911800 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 816/1631] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:04.911800 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70349299-291c-4667-bd83-3a21f9ae0146 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:04.917382 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:04.941863 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 816/1632] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:04.942067 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adca9dd-ba35-4f14-8854-a18487b65ee8 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:04.971174 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adca9dd-ba35-4f14-8854-a18487b65ee8 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:04.981631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35985019-7976-4d02-80f0-fa28b9ea7c8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:04.982355 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35985019-7976-4d02-80f0-fa28b9ea7c8e service nova] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:04.984793 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35985019-7976-4d02-80f0-fa28b9ea7c8e service nova] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:04.985077 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35985019-7976-4d02-80f0-fa28b9ea7c8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:05.001521 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70349299-291c-4667-bd83-3a21f9ae0146 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:05.006113 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4899f8-912d-4663-8083-f8975f904104 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:05.007045 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4899f8-912d-4663-8083-f8975f904104 service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:05.007370 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4899f8-912d-4663-8083-f8975f904104 service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:05.007642 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4899f8-912d-4663-8083-f8975f904104 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:05.015881 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:05.021649 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4899f8-912d-4663-8083-f8975f904104 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:05.023282 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4899f8-912d-4663-8083-f8975f904104 service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:05.035479 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 817/1633] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:04 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:05.036287 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4899f8-912d-4663-8083-f8975f904104 service cinder] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:05.054397 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4899f8-912d-4663-8083-f8975f904104 service cinder] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:05.063154 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a34b3-9901-4164-893a-4e8d735299db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:05.063976 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a34b3-9901-4164-893a-4e8d735299db admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:05.064274 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a34b3-9901-4164-893a-4e8d735299db admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:05.064585 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a34b3-9901-4164-893a-4e8d735299db admin admin] PATH_INFO: `/v3/projects/efedff4d28bf468dbd796b113e134470/users/5f01710a590343e4ab30bba3bda0ea42/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:05.112891 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1a34b3-9901-4164-893a-4e8d735299db admin admin] RBAC: Authorizing `identity:create_grant(project_id=efedff4d28bf468dbd796b113e134470, user_id=5f01710a590343e4ab30bba3bda0ea42, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:05.121589 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1a34b3-9901-4164-893a-4e8d735299db admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:05.152647 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35985019-7976-4d02-80f0-fa28b9ea7c8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:05.163604 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 818/1634] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:05 2022] PUT /identity/v3/projects/efedff4d28bf468dbd796b113e134470/users/5f01710a590343e4ab30bba3bda0ea42/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:05.164850 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a34b3-9901-4164-893a-4e8d735299db admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:05.167454 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35985019-7976-4d02-80f0-fa28b9ea7c8e service nova] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:05.201784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 817/1635] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:05.204113 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35985019-7976-4d02-80f0-fa28b9ea7c8e service nova] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:05.250281 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35985019-7976-4d02-80f0-fa28b9ea7c8e service nova] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:05.351486 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a34b3-9901-4164-893a-4e8d735299db admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:05.364981 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784248fa-9e36-4d3d-9499-fd5ef1df0bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:05.365675 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784248fa-9e36-4d3d-9499-fd5ef1df0bf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:05.369412 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784248fa-9e36-4d3d-9499-fd5ef1df0bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:05.369779 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784248fa-9e36-4d3d-9499-fd5ef1df0bf5 admin admin] PATH_INFO: `/v3/projects/0de544bee8754ed6bd7815538515f1f7/users/1276c81b1a884368b7bb884948fb07d3/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:05.392018 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68b5d5-3a47-48d2-b8bb-1b8fc83b0bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:05.395708 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68b5d5-3a47-48d2-b8bb-1b8fc83b0bdf admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:05.397128 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68b5d5-3a47-48d2-b8bb-1b8fc83b0bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:05.397463 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68b5d5-3a47-48d2-b8bb-1b8fc83b0bdf admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:05.398539 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc68b5d5-3a47-48d2-b8bb-1b8fc83b0bdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:05.403514 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc68b5d5-3a47-48d2-b8bb-1b8fc83b0bdf admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:05.410840 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784248fa-9e36-4d3d-9499-fd5ef1df0bf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0de544bee8754ed6bd7815538515f1f7, user_id=1276c81b1a884368b7bb884948fb07d3, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:05.418621 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784248fa-9e36-4d3d-9499-fd5ef1df0bf5 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:05.431393 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 818/1636] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:05.432603 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68b5d5-3a47-48d2-b8bb-1b8fc83b0bdf admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:05.467738 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 819/1637] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:05 2022] PUT /identity/v3/projects/0de544bee8754ed6bd7815538515f1f7/users/1276c81b1a884368b7bb884948fb07d3/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:05.474126 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eba7f0-801b-43a5-9337-73527df950bb None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:05.474416 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eba7f0-801b-43a5-9337-73527df950bb None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:05.477321 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eba7f0-801b-43a5-9337-73527df950bb None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:05.481364 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 820/1638] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 14:26:05 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:26:05.484199 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eba7f0-801b-43a5-9337-73527df950bb None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:05.599643 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68b5d5-3a47-48d2-b8bb-1b8fc83b0bdf admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:05.661811 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eba7f0-801b-43a5-9337-73527df950bb None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:05.671616 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7e54af-d5a4-46b8-8c07-7d6fb693a777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:05.672822 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e54af-d5a4-46b8-8c07-7d6fb693a777 admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:05.673741 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e54af-d5a4-46b8-8c07-7d6fb693a777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:05.675326 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e54af-d5a4-46b8-8c07-7d6fb693a777 admin admin] PATH_INFO: `/v3/projects/efedff4d28bf468dbd796b113e134470/users/5f01710a590343e4ab30bba3bda0ea42/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:05.678222 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10161de1-3d18-4f72-91ee-113b854414a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:05.678867 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10161de1-3d18-4f72-91ee-113b854414a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:05.679256 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10161de1-3d18-4f72-91ee-113b854414a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:05.679481 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7e54af-d5a4-46b8-8c07-7d6fb693a777 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efedff4d28bf468dbd796b113e134470, user_id=5f01710a590343e4ab30bba3bda0ea42, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:05.679589 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10161de1-3d18-4f72-91ee-113b854414a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:05.682921 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7e54af-d5a4-46b8-8c07-7d6fb693a777 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:05.692890 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:05.717757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 821/1639] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:05 2022] PUT /identity/v3/projects/efedff4d28bf468dbd796b113e134470/users/5f01710a590343e4ab30bba3bda0ea42/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:05.717757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7e54af-d5a4-46b8-8c07-7d6fb693a777 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:05.720156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:05.779037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10161de1-3d18-4f72-91ee-113b854414a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:05.780520 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10161de1-3d18-4f72-91ee-113b854414a7 service cinder] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:05.794558 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 819/1640] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:05 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:05.796020 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660b933-11c6-4ffd-8bc1-9dbe4e05b63b None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:05.796268 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660b933-11c6-4ffd-8bc1-9dbe4e05b63b None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:05.796524 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660b933-11c6-4ffd-8bc1-9dbe4e05b63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:05.822573 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-7660b933-11c6-4ffd-8bc1-9dbe4e05b63b None None] MFA Rules not processed for user `79eaaae97aac4282a613939e5b768376`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:05.823833 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-7660b933-11c6-4ffd-8bc1-9dbe4e05b63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:05.858153 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7e54af-d5a4-46b8-8c07-7d6fb693a777 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:05.879001 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19936427-7c1c-4a33-a7cd-cf8f9b7057c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:05.880008 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19936427-7c1c-4a33-a7cd-cf8f9b7057c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:05.880311 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19936427-7c1c-4a33-a7cd-cf8f9b7057c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:05.881335 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19936427-7c1c-4a33-a7cd-cf8f9b7057c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:05.881634 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19936427-7c1c-4a33-a7cd-cf8f9b7057c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:05.884261 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19936427-7c1c-4a33-a7cd-cf8f9b7057c4 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:05.910874 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 822/1641] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:05.910874 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b17abd-6825-4f24-b8af-a316fe036bee None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:05.910874 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b17abd-6825-4f24-b8af-a316fe036bee None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:05.910874 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b17abd-6825-4f24-b8af-a316fe036bee None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:05.917479 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 823/1642] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 14:26:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:26:05.937800 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea1285-7651-459a-b8ce-de025880a0c3 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:05.938030 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea1285-7651-459a-b8ce-de025880a0c3 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:05.938613 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea1285-7651-459a-b8ce-de025880a0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:05.940404 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 820/1643] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 14:26:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:05.947469 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660b933-11c6-4ffd-8bc1-9dbe4e05b63b None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:05.966925 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660b933-11c6-4ffd-8bc1-9dbe4e05b63b None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:05.975570 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7fc9d1-51fa-4648-b336-cffd661f85ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:05.976348 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7fc9d1-51fa-4648-b336-cffd661f85ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:05.976686 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7fc9d1-51fa-4648-b336-cffd661f85ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:05.976958 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7fc9d1-51fa-4648-b336-cffd661f85ca admin admin] PATH_INFO: `/v3/projects/0de544bee8754ed6bd7815538515f1f7/users/1276c81b1a884368b7bb884948fb07d3/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:05.981905 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7fc9d1-51fa-4648-b336-cffd661f85ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=0de544bee8754ed6bd7815538515f1f7, user_id=1276c81b1a884368b7bb884948fb07d3, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:05.984819 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7fc9d1-51fa-4648-b336-cffd661f85ca admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:05.999089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-29ea1285-7651-459a-b8ce-de025880a0c3 None None] MFA Rules not processed for user `5f01710a590343e4ab30bba3bda0ea42`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:06.000463 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-29ea1285-7651-459a-b8ce-de025880a0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:06.027054 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 821/1644] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:05 2022] PUT /identity/v3/projects/0de544bee8754ed6bd7815538515f1f7/users/1276c81b1a884368b7bb884948fb07d3/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:06.027054 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8640a-4541-4d8a-8afd-1bdd0e1464c3 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:06.027054 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8640a-4541-4d8a-8afd-1bdd0e1464c3 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:06.027054 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8640a-4541-4d8a-8afd-1bdd0e1464c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:06.072488 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-08e8640a-4541-4d8a-8afd-1bdd0e1464c3 None None] MFA Rules not processed for user `79eaaae97aac4282a613939e5b768376`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:06.073928 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-08e8640a-4541-4d8a-8afd-1bdd0e1464c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:06.228096 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 824/1645] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:05 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:06.228096 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea1285-7651-459a-b8ce-de025880a0c3 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:06.264632 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 822/1646] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 14:26:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:06.266544 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3becfd0e-31fc-403a-95cd-15144ee1534f None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:06.266838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3becfd0e-31fc-403a-95cd-15144ee1534f None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:06.269048 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3becfd0e-31fc-403a-95cd-15144ee1534f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:06.335051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea1285-7651-459a-b8ce-de025880a0c3 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:06.341534 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6fa572-0e7b-4d3f-8247-f049bdf7873d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:06.342172 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6fa572-0e7b-4d3f-8247-f049bdf7873d service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:06.342589 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6fa572-0e7b-4d3f-8247-f049bdf7873d service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:06.342888 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6fa572-0e7b-4d3f-8247-f049bdf7873d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:06.366916 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6fa572-0e7b-4d3f-8247-f049bdf7873d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:06.369230 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6fa572-0e7b-4d3f-8247-f049bdf7873d service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:06.377205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-3becfd0e-31fc-403a-95cd-15144ee1534f None None] MFA Rules not processed for user `1276c81b1a884368b7bb884948fb07d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:06.378378 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-3becfd0e-31fc-403a-95cd-15144ee1534f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:06.396976 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 825/1647] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:06.397953 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6fa572-0e7b-4d3f-8247-f049bdf7873d service cinder] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:06.409768 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6fa572-0e7b-4d3f-8247-f049bdf7873d service cinder] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:06.414005 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6773d-6ea2-4853-a8a3-2a093ce39830 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:06.414234 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6773d-6ea2-4853-a8a3-2a093ce39830 service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:06.414501 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6773d-6ea2-4853-a8a3-2a093ce39830 service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:06.414968 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6773d-6ea2-4853-a8a3-2a093ce39830 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:06.421752 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:06.424693 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa6773d-6ea2-4853-a8a3-2a093ce39830 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:06.426345 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa6773d-6ea2-4853-a8a3-2a093ce39830 service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:06.438167 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 826/1648] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:06 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:06.508821 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6773d-6ea2-4853-a8a3-2a093ce39830 service cinder] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:06.517833 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6773d-6ea2-4853-a8a3-2a093ce39830 service cinder] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:06.541888 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8a01f8-350e-4bf9-b4dc-7ede1bbbccd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:06.541888 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a01f8-350e-4bf9-b4dc-7ede1bbbccd2 service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:06.541888 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a01f8-350e-4bf9-b4dc-7ede1bbbccd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:06.541888 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a01f8-350e-4bf9-b4dc-7ede1bbbccd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:06.573005 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:06.576391 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8a01f8-350e-4bf9-b4dc-7ede1bbbccd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:06.578296 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8a01f8-350e-4bf9-b4dc-7ede1bbbccd2 service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:06.610678 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 827/1649] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 14:26:06.622268 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 823/1650] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:06 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:06.721245 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8a01f8-350e-4bf9-b4dc-7ede1bbbccd2 service cinder] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:06.732668 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8a01f8-350e-4bf9-b4dc-7ede1bbbccd2 service cinder] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:06.742204 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14b6a5-5f32-4882-a225-8a6a57569fa1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:06.743427 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14b6a5-5f32-4882-a225-8a6a57569fa1 service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:06.743799 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14b6a5-5f32-4882-a225-8a6a57569fa1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:06.744353 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14b6a5-5f32-4882-a225-8a6a57569fa1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:06.768550 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b14b6a5-5f32-4882-a225-8a6a57569fa1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:06.770091 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b14b6a5-5f32-4882-a225-8a6a57569fa1 service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:06.795184 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 828/1651] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:06 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:07.205152 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3becfd0e-31fc-403a-95cd-15144ee1534f None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:07.228990 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3becfd0e-31fc-403a-95cd-15144ee1534f None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:07.245577 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7a17f-7ab8-43ee-88da-71b7cfb847b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:07.246229 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7a17f-7ab8-43ee-88da-71b7cfb847b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:07.248232 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7a17f-7ab8-43ee-88da-71b7cfb847b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:07.248468 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7a17f-7ab8-43ee-88da-71b7cfb847b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:07.401529 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b7a17f-7ab8-43ee-88da-71b7cfb847b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:07.401529 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b7a17f-7ab8-43ee-88da-71b7cfb847b6 service cinder] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:07.405330 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 824/1652] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:07 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:07.789222 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14b6a5-5f32-4882-a225-8a6a57569fa1 service cinder] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:07.950520 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14b6a5-5f32-4882-a225-8a6a57569fa1 service cinder] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:07.958058 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e52cdc-c4df-4180-b533-142f9d45aad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:07.959017 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e52cdc-c4df-4180-b533-142f9d45aad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:07.959549 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e52cdc-c4df-4180-b533-142f9d45aad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:07.960330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e52cdc-c4df-4180-b533-142f9d45aad5 admin admin] PATH_INFO: `/v3/users/2038b90f689442489fb00fc6761d28c0` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:07.961966 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e52cdc-c4df-4180-b533-142f9d45aad5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2038b90f689442489fb00fc6761d28c0)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:07.966135 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e52cdc-c4df-4180-b533-142f9d45aad5 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:08.123418 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-28e52cdc-c4df-4180-b533-142f9d45aad5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2038b90f689442489fb00fc6761d28c0'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:08.124118 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-28e52cdc-c4df-4180-b533-142f9d45aad5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2038b90f689442489fb00fc6761d28c0'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:08.126948 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-28e52cdc-c4df-4180-b533-142f9d45aad5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2038b90f689442489fb00fc6761d28c0'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:08.154210 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-28e52cdc-c4df-4180-b533-142f9d45aad5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2038b90f689442489fb00fc6761d28c0'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:08.169115 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-28e52cdc-c4df-4180-b533-142f9d45aad5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2038b90f689442489fb00fc6761d28c0'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:08.186135 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7a17f-7ab8-43ee-88da-71b7cfb847b6 service cinder] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:08.204025 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7a17f-7ab8-43ee-88da-71b7cfb847b6 service cinder] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:08.207868 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 829/1653] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:26:07 2022] DELETE /identity/v3/users/2038b90f689442489fb00fc6761d28c0 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:08.215465 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd135add-28ce-4171-a878-073f978689d3 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:08.215818 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd135add-28ce-4171-a878-073f978689d3 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:08.216098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd135add-28ce-4171-a878-073f978689d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:08.263065 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-bd135add-28ce-4171-a878-073f978689d3 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:08.264317 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-bd135add-28ce-4171-a878-073f978689d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:08.355740 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856933b-af6e-47b6-89a8-63eed7221d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:08.356896 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856933b-af6e-47b6-89a8-63eed7221d58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:08.357280 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856933b-af6e-47b6-89a8-63eed7221d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:08.357690 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856933b-af6e-47b6-89a8-63eed7221d58 admin admin] PATH_INFO: `/v3/users/065701e193524356bf2846785eebf68e` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:08.359957 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2856933b-af6e-47b6-89a8-63eed7221d58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=065701e193524356bf2846785eebf68e)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:08.363122 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: WARNING py.warnings [None req-2856933b-af6e-47b6-89a8-63eed7221d58 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 14:26:08.363122 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: warnings.warn(msg) Sep 24 14:26:08.363122 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: Sep 24 14:26:08.365496 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2856933b-af6e-47b6-89a8-63eed7221d58 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:08.423037 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 830/1654] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:08.491113 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd135add-28ce-4171-a878-073f978689d3 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:08.493137 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-bd135add-28ce-4171-a878-073f978689d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:08.499101 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:08.590516 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-2856933b-af6e-47b6-89a8-63eed7221d58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '065701e193524356bf2846785eebf68e'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:08.594592 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-2856933b-af6e-47b6-89a8-63eed7221d58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '065701e193524356bf2846785eebf68e'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:08.615472 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd135add-28ce-4171-a878-073f978689d3 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:08.621410 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbac9319-4b0f-4a87-817b-2b44b239602b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:08.622042 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac9319-4b0f-4a87-817b-2b44b239602b service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:08.622324 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac9319-4b0f-4a87-817b-2b44b239602b service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:08.622600 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac9319-4b0f-4a87-817b-2b44b239602b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:08.628775 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-cbac9319-4b0f-4a87-817b-2b44b239602b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:08.652686 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-2856933b-af6e-47b6-89a8-63eed7221d58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '065701e193524356bf2846785eebf68e'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:08.653736 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-2856933b-af6e-47b6-89a8-63eed7221d58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '065701e193524356bf2846785eebf68e'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:08.654288 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbac9319-4b0f-4a87-817b-2b44b239602b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:08.655588 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:08.661047 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbac9319-4b0f-4a87-817b-2b44b239602b service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:08.690904 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 831/1655] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 14:26:08.721138 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-2856933b-af6e-47b6-89a8-63eed7221d58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '065701e193524356bf2846785eebf68e'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:08.771839 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 825/1656] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:26:08 2022] DELETE /identity/v3/users/065701e193524356bf2846785eebf68e => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:26:08.776829 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a916ef-a1f1-418f-9eef-8f37e8e26a6c None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:08.777119 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a916ef-a1f1-418f-9eef-8f37e8e26a6c None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:08.777396 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a916ef-a1f1-418f-9eef-8f37e8e26a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:08.817496 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-64a916ef-a1f1-418f-9eef-8f37e8e26a6c None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:08.818767 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-64a916ef-a1f1-418f-9eef-8f37e8e26a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:08.929327 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 832/1657] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:08.996551 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856933b-af6e-47b6-89a8-63eed7221d58 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:08.998178 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-2856933b-af6e-47b6-89a8-63eed7221d58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:09.253224 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856933b-af6e-47b6-89a8-63eed7221d58 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:09.268205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d0bca7-1176-4e2d-b407-efd57f787b94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:09.271291 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d0bca7-1176-4e2d-b407-efd57f787b94 service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:09.272057 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d0bca7-1176-4e2d-b407-efd57f787b94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:09.273359 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d0bca7-1176-4e2d-b407-efd57f787b94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:09.312554 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d0bca7-1176-4e2d-b407-efd57f787b94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:09.316965 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d0bca7-1176-4e2d-b407-efd57f787b94 service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:09.357042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 826/1658] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:11.029069 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a916ef-a1f1-418f-9eef-8f37e8e26a6c None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:11.033025 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-64a916ef-a1f1-418f-9eef-8f37e8e26a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:11.093103 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a916ef-a1f1-418f-9eef-8f37e8e26a6c None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:11.122649 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdcb01a-8d5a-4f70-902b-40efdce94669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:11.126584 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdcb01a-8d5a-4f70-902b-40efdce94669 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:11.126903 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdcb01a-8d5a-4f70-902b-40efdce94669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:11.127189 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdcb01a-8d5a-4f70-902b-40efdce94669 admin admin] PATH_INFO: `/v3/projects/d05d72126e6d4ec58cee3cc804a93fb6` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:11.130594 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdcb01a-8d5a-4f70-902b-40efdce94669 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d05d72126e6d4ec58cee3cc804a93fb6)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:11.131785 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: WARNING py.warnings [None req-7cdcb01a-8d5a-4f70-902b-40efdce94669 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 14:26:11.131785 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: warnings.warn(msg) Sep 24 14:26:11.131785 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: Sep 24 14:26:11.135203 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdcb01a-8d5a-4f70-902b-40efdce94669 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:11.211927 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d0bca7-1176-4e2d-b407-efd57f787b94 service neutron] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:11.214376 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-b8d0bca7-1176-4e2d-b407-efd57f787b94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:11.240083 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d0bca7-1176-4e2d-b407-efd57f787b94 service neutron] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:11.255031 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fade591-4259-4df2-a602-634e41db0982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:11.256311 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fade591-4259-4df2-a602-634e41db0982 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:11.256793 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fade591-4259-4df2-a602-634e41db0982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:11.257309 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fade591-4259-4df2-a602-634e41db0982 admin admin] PATH_INFO: `/v3/users/2d15654b29bd450f9af8e39b9119e3fa` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:11.259689 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fade591-4259-4df2-a602-634e41db0982 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d15654b29bd450f9af8e39b9119e3fa)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:11.266741 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fade591-4259-4df2-a602-634e41db0982 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:11.373646 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-3fade591-4259-4df2-a602-634e41db0982 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d15654b29bd450f9af8e39b9119e3fa'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:11.374532 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-3fade591-4259-4df2-a602-634e41db0982 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d15654b29bd450f9af8e39b9119e3fa'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:11.387216 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-3fade591-4259-4df2-a602-634e41db0982 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d15654b29bd450f9af8e39b9119e3fa'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:11.388753 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-3fade591-4259-4df2-a602-634e41db0982 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d15654b29bd450f9af8e39b9119e3fa'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:11.396793 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-7cdcb01a-8d5a-4f70-902b-40efdce94669 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd05d72126e6d4ec58cee3cc804a93fb6', 'request_id': 'req-7cdcb01a-8d5a-4f70-902b-40efdce94669'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:11.436694 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-3fade591-4259-4df2-a602-634e41db0982 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d15654b29bd450f9af8e39b9119e3fa'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:11.483680 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-7cdcb01a-8d5a-4f70-902b-40efdce94669 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd05d72126e6d4ec58cee3cc804a93fb6', 'request_id': 'req-7cdcb01a-8d5a-4f70-902b-40efdce94669'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:11.502120 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-7cdcb01a-8d5a-4f70-902b-40efdce94669 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd05d72126e6d4ec58cee3cc804a93fb6', 'request_id': 'req-7cdcb01a-8d5a-4f70-902b-40efdce94669'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:11.514979 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 827/1659] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:26:11 2022] DELETE /identity/v3/users/2d15654b29bd450f9af8e39b9119e3fa => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:11.521295 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f19a9-dec2-4fbf-a154-ef2425b063ad None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:11.521738 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f19a9-dec2-4fbf-a154-ef2425b063ad None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:11.522087 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f19a9-dec2-4fbf-a154-ef2425b063ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:11.549805 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-7cdcb01a-8d5a-4f70-902b-40efdce94669 admin admin] The token cache is being invalidate because project d05d72126e6d4ec58cee3cc804a93fb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:26:11.550326 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-7cdcb01a-8d5a-4f70-902b-40efdce94669 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:11.552814 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-7cdcb01a-8d5a-4f70-902b-40efdce94669 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:11.560501 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-681f19a9-dec2-4fbf-a154-ef2425b063ad None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:11.562084 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-681f19a9-dec2-4fbf-a154-ef2425b063ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:11.562777 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 833/1660] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:26:11 2022] DELETE /identity/v3/projects/d05d72126e6d4ec58cee3cc804a93fb6 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:11.567454 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b59e6-820e-479b-9e52-dd2080ac9d26 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:11.567742 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b59e6-820e-479b-9e52-dd2080ac9d26 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:11.568018 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b59e6-820e-479b-9e52-dd2080ac9d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:11.642969 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-472b59e6-820e-479b-9e52-dd2080ac9d26 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:11.655289 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-472b59e6-820e-479b-9e52-dd2080ac9d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:11.745114 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:11.758386 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:11.804673 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 828/1661] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:11.921981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d35f1-29e6-4c32-9a5a-1cda8f536567 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:11.922259 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d35f1-29e6-4c32-9a5a-1cda8f536567 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:11.922469 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 834/1662] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:11.922858 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d35f1-29e6-4c32-9a5a-1cda8f536567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:11.972990 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-c62d35f1-29e6-4c32-9a5a-1cda8f536567 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:11.974185 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-c62d35f1-29e6-4c32-9a5a-1cda8f536567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:12.001973 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 829/1663] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:12.005886 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b59e6-820e-479b-9e52-dd2080ac9d26 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:12.033820 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b59e6-820e-479b-9e52-dd2080ac9d26 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:12.050629 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62d35f1-29e6-4c32-9a5a-1cda8f536567 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:12.053409 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bffb4ce-50e9-4325-9042-370c467baa3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:12.054137 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-c62d35f1-29e6-4c32-9a5a-1cda8f536567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:12.054411 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bffb4ce-50e9-4325-9042-370c467baa3b admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:12.054884 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bffb4ce-50e9-4325-9042-370c467baa3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:12.055298 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bffb4ce-50e9-4325-9042-370c467baa3b admin admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:12.056091 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bffb4ce-50e9-4325-9042-370c467baa3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:26:12.056534 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bffb4ce-50e9-4325-9042-370c467baa3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:12.062838 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bffb4ce-50e9-4325-9042-370c467baa3b admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:12.075979 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62d35f1-29e6-4c32-9a5a-1cda8f536567 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:12.094774 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad76ffe-1a2a-4e0c-9b8d-cc3b588b4b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:12.096112 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad76ffe-1a2a-4e0c-9b8d-cc3b588b4b77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:12.096557 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad76ffe-1a2a-4e0c-9b8d-cc3b588b4b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:12.096966 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad76ffe-1a2a-4e0c-9b8d-cc3b588b4b77 admin admin] PATH_INFO: `/v3/projects/e25e868cd2d44dfe964975ce9268b576` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:12.098770 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad76ffe-1a2a-4e0c-9b8d-cc3b588b4b77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e25e868cd2d44dfe964975ce9268b576)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:12.103848 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad76ffe-1a2a-4e0c-9b8d-cc3b588b4b77 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:12.126006 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 835/1664] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:26:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:26:12.127242 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bffb4ce-50e9-4325-9042-370c467baa3b admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:12.129316 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-6bffb4ce-50e9-4325-9042-370c467baa3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:12.262855 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-1ad76ffe-1a2a-4e0c-9b8d-cc3b588b4b77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e25e868cd2d44dfe964975ce9268b576', 'request_id': 'req-1ad76ffe-1a2a-4e0c-9b8d-cc3b588b4b77'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:12.272244 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-1ad76ffe-1a2a-4e0c-9b8d-cc3b588b4b77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e25e868cd2d44dfe964975ce9268b576', 'request_id': 'req-1ad76ffe-1a2a-4e0c-9b8d-cc3b588b4b77'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:12.290797 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-1ad76ffe-1a2a-4e0c-9b8d-cc3b588b4b77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e25e868cd2d44dfe964975ce9268b576', 'request_id': 'req-1ad76ffe-1a2a-4e0c-9b8d-cc3b588b4b77'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:12.332053 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bffb4ce-50e9-4325-9042-370c467baa3b admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:12.332989 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-1ad76ffe-1a2a-4e0c-9b8d-cc3b588b4b77 admin admin] The token cache is being invalidate because project e25e868cd2d44dfe964975ce9268b576 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:26:12.333780 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-1ad76ffe-1a2a-4e0c-9b8d-cc3b588b4b77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:12.335361 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-1ad76ffe-1a2a-4e0c-9b8d-cc3b588b4b77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:12.341734 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 830/1665] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:26:12 2022] DELETE /identity/v3/projects/e25e868cd2d44dfe964975ce9268b576 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:12.351546 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad76ffe-1a2a-4e0c-9b8d-cc3b588b4b77 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:12.359100 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-1ad76ffe-1a2a-4e0c-9b8d-cc3b588b4b77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:12.457848 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc05b6e-61bc-4cd8-90d9-05e435a65ec3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:12.458538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc05b6e-61bc-4cd8-90d9-05e435a65ec3 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:12.458768 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc05b6e-61bc-4cd8-90d9-05e435a65ec3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:12.459072 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc05b6e-61bc-4cd8-90d9-05e435a65ec3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:12.461933 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-abc05b6e-61bc-4cd8-90d9-05e435a65ec3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:12.585438 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad76ffe-1a2a-4e0c-9b8d-cc3b588b4b77 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:12.628209 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81cb723-f05b-4e87-a6d9-ef6c937649c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:12.631208 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81cb723-f05b-4e87-a6d9-ef6c937649c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:12.631507 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81cb723-f05b-4e87-a6d9-ef6c937649c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:12.631927 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81cb723-f05b-4e87-a6d9-ef6c937649c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:12.632445 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81cb723-f05b-4e87-a6d9-ef6c937649c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:12.635474 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc05b6e-61bc-4cd8-90d9-05e435a65ec3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:12.637944 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc05b6e-61bc-4cd8-90d9-05e435a65ec3 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:12.645775 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81cb723-f05b-4e87-a6d9-ef6c937649c7 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:12.653645 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 836/1666] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:12.654631 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc05b6e-61bc-4cd8-90d9-05e435a65ec3 service neutron] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:12.656102 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-abc05b6e-61bc-4cd8-90d9-05e435a65ec3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:12.681219 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc05b6e-61bc-4cd8-90d9-05e435a65ec3 service neutron] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:12.691168 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20df2e3b-137b-41ba-83b8-d404a4e8d922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:12.691842 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df2e3b-137b-41ba-83b8-d404a4e8d922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:12.692154 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df2e3b-137b-41ba-83b8-d404a4e8d922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:12.692433 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df2e3b-137b-41ba-83b8-d404a4e8d922 admin admin] PATH_INFO: `/v3/users/1276c81b1a884368b7bb884948fb07d3` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:12.694410 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20df2e3b-137b-41ba-83b8-d404a4e8d922 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1276c81b1a884368b7bb884948fb07d3)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:12.699493 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20df2e3b-137b-41ba-83b8-d404a4e8d922 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:12.748200 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:12.753333 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 831/1667] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:26:12 2022] POST /identity/v3/projects => generated 403 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:12.757522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81cb723-f05b-4e87-a6d9-ef6c937649c7 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:12.850290 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-20df2e3b-137b-41ba-83b8-d404a4e8d922 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1276c81b1a884368b7bb884948fb07d3'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:12.851938 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-20df2e3b-137b-41ba-83b8-d404a4e8d922 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1276c81b1a884368b7bb884948fb07d3'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:12.852793 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-20df2e3b-137b-41ba-83b8-d404a4e8d922 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1276c81b1a884368b7bb884948fb07d3'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:12.892362 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-20df2e3b-137b-41ba-83b8-d404a4e8d922 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1276c81b1a884368b7bb884948fb07d3'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:12.906719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-20df2e3b-137b-41ba-83b8-d404a4e8d922 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1276c81b1a884368b7bb884948fb07d3'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:12.941686 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 837/1668] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:26:12 2022] DELETE /identity/v3/users/1276c81b1a884368b7bb884948fb07d3 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:12.959085 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:12.988734 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81cb723-f05b-4e87-a6d9-ef6c937649c7 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:13.098861 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c07ca8-00e9-4d12-bf82-9f8d298af79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:13.100145 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c07ca8-00e9-4d12-bf82-9f8d298af79b admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:13.100681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c07ca8-00e9-4d12-bf82-9f8d298af79b admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:13.102279 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c07ca8-00e9-4d12-bf82-9f8d298af79b admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:13.105037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c07ca8-00e9-4d12-bf82-9f8d298af79b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:13.110360 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c07ca8-00e9-4d12-bf82-9f8d298af79b admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:13.161704 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 832/1669] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:26:12 2022] POST /identity/v3/users => generated 302 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:13.166098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20df2e3b-137b-41ba-83b8-d404a4e8d922 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:13.167683 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-20df2e3b-137b-41ba-83b8-d404a4e8d922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:13.192727 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20df2e3b-137b-41ba-83b8-d404a4e8d922 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:13.216892 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7184d2-34d4-4c64-8843-5c7c0350fb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:13.217718 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7184d2-34d4-4c64-8843-5c7c0350fb6c admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:13.218139 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7184d2-34d4-4c64-8843-5c7c0350fb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:13.218653 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7184d2-34d4-4c64-8843-5c7c0350fb6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:13.219520 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7184d2-34d4-4c64-8843-5c7c0350fb6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:13.221299 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7184d2-34d4-4c64-8843-5c7c0350fb6c admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:13.239980 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 838/1670] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:13.246302 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c07ca8-00e9-4d12-bf82-9f8d298af79b admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:13.275538 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c07ca8-00e9-4d12-bf82-9f8d298af79b admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:13.286156 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7184d2-34d4-4c64-8843-5c7c0350fb6c admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:13.288460 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-fd7184d2-34d4-4c64-8843-5c7c0350fb6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:13.294387 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7dd485-7a0d-4d08-9b66-118bd934819b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:13.295320 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7dd485-7a0d-4d08-9b66-118bd934819b admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:13.295760 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7dd485-7a0d-4d08-9b66-118bd934819b admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:13.296165 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7dd485-7a0d-4d08-9b66-118bd934819b admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:13.297135 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7dd485-7a0d-4d08-9b66-118bd934819b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:13.299268 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7dd485-7a0d-4d08-9b66-118bd934819b admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:13.314872 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7184d2-34d4-4c64-8843-5c7c0350fb6c admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:13.321136 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c298f-9119-49f4-a557-34df24de80af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:13.322238 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c298f-9119-49f4-a557-34df24de80af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:13.322752 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c298f-9119-49f4-a557-34df24de80af admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:13.323210 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c298f-9119-49f4-a557-34df24de80af admin admin] PATH_INFO: `/v3/projects/c8d85d19f92d43d38b05bf9a5a73f550` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:13.324810 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482c298f-9119-49f4-a557-34df24de80af admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8d85d19f92d43d38b05bf9a5a73f550)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:13.326946 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482c298f-9119-49f4-a557-34df24de80af admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:13.350373 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 833/1671] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:13.352786 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7dd485-7a0d-4d08-9b66-118bd934819b admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:13.371629 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7dd485-7a0d-4d08-9b66-118bd934819b admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:13.378276 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d525415-1c7f-431e-b422-eb74e710584f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:13.379306 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d525415-1c7f-431e-b422-eb74e710584f admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:13.379811 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d525415-1c7f-431e-b422-eb74e710584f admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:13.380907 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d525415-1c7f-431e-b422-eb74e710584f admin admin] PATH_INFO: `/v3/projects/97bf63ef39ef400ab646dd447451aae2/users/6c14990386754fd79905a63dd2606eb0/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:13.399112 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-482c298f-9119-49f4-a557-34df24de80af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8d85d19f92d43d38b05bf9a5a73f550', 'request_id': 'req-482c298f-9119-49f4-a557-34df24de80af'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:13.407446 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-482c298f-9119-49f4-a557-34df24de80af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8d85d19f92d43d38b05bf9a5a73f550', 'request_id': 'req-482c298f-9119-49f4-a557-34df24de80af'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:13.416508 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d525415-1c7f-431e-b422-eb74e710584f admin admin] RBAC: Authorizing `identity:create_grant(project_id=97bf63ef39ef400ab646dd447451aae2, user_id=6c14990386754fd79905a63dd2606eb0, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:13.419871 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d525415-1c7f-431e-b422-eb74e710584f admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:13.421859 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-482c298f-9119-49f4-a557-34df24de80af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8d85d19f92d43d38b05bf9a5a73f550', 'request_id': 'req-482c298f-9119-49f4-a557-34df24de80af'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:13.438221 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-482c298f-9119-49f4-a557-34df24de80af admin admin] The token cache is being invalidate because project c8d85d19f92d43d38b05bf9a5a73f550 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:26:13.438919 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-482c298f-9119-49f4-a557-34df24de80af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:13.440234 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-482c298f-9119-49f4-a557-34df24de80af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:13.441795 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 834/1672] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:13 2022] PUT /identity/v3/projects/97bf63ef39ef400ab646dd447451aae2/users/6c14990386754fd79905a63dd2606eb0/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:13.442531 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 839/1673] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:26:13 2022] DELETE /identity/v3/projects/c8d85d19f92d43d38b05bf9a5a73f550 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:13.448921 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4850fee-214a-4283-ac66-25c9ae628d4e None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:13.449207 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4850fee-214a-4283-ac66-25c9ae628d4e None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:13.449303 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bcaa6e-8c84-425c-a2c0-bda6da3ae0f0 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:13.449487 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4850fee-214a-4283-ac66-25c9ae628d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:13.449580 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bcaa6e-8c84-425c-a2c0-bda6da3ae0f0 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:13.449840 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bcaa6e-8c84-425c-a2c0-bda6da3ae0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:13.481906 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-f4850fee-214a-4283-ac66-25c9ae628d4e None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:13.484334 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-f4850fee-214a-4283-ac66-25c9ae628d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:13.489289 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-09bcaa6e-8c84-425c-a2c0-bda6da3ae0f0 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:13.490900 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-09bcaa6e-8c84-425c-a2c0-bda6da3ae0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:13.575049 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 835/1674] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:13.580480 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a58d65-79d4-4962-a2aa-fa89294b3c60 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:13.582261 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a58d65-79d4-4962-a2aa-fa89294b3c60 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:13.583013 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a58d65-79d4-4962-a2aa-fa89294b3c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:13.600438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 840/1675] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:13.612190 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4850fee-214a-4283-ac66-25c9ae628d4e None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:13.613619 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-f4850fee-214a-4283-ac66-25c9ae628d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:13.624412 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-82a58d65-79d4-4962-a2aa-fa89294b3c60 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:13.625979 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-82a58d65-79d4-4962-a2aa-fa89294b3c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:13.632083 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:13.644458 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 836/1676] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:13.654527 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a58d65-79d4-4962-a2aa-fa89294b3c60 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:13.670401 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a58d65-79d4-4962-a2aa-fa89294b3c60 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:13.691639 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1549a2f-5314-45e4-a8b8-3f8523d3bf5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:13.692259 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1549a2f-5314-45e4-a8b8-3f8523d3bf5b admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:13.692535 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1549a2f-5314-45e4-a8b8-3f8523d3bf5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:13.692805 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1549a2f-5314-45e4-a8b8-3f8523d3bf5b admin admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:13.697440 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1549a2f-5314-45e4-a8b8-3f8523d3bf5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:26:13.697440 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1549a2f-5314-45e4-a8b8-3f8523d3bf5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:13.698267 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1549a2f-5314-45e4-a8b8-3f8523d3bf5b admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:13.722309 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4850fee-214a-4283-ac66-25c9ae628d4e None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:13.727098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 837/1677] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:26:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:26:13.733402 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30136c2f-0759-49b8-92b5-e87e9c5c8cac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:13.735198 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1549a2f-5314-45e4-a8b8-3f8523d3bf5b admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:13.735778 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30136c2f-0759-49b8-92b5-e87e9c5c8cac service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:13.736073 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30136c2f-0759-49b8-92b5-e87e9c5c8cac service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:13.736341 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30136c2f-0759-49b8-92b5-e87e9c5c8cac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:13.758879 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1549a2f-5314-45e4-a8b8-3f8523d3bf5b admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:13.764065 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30136c2f-0759-49b8-92b5-e87e9c5c8cac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:13.765974 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30136c2f-0759-49b8-92b5-e87e9c5c8cac service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:13.775935 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cac4e8-723b-4b9f-aeb5-73b3ce35d1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:13.776628 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cac4e8-723b-4b9f-aeb5-73b3ce35d1ce admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:13.776896 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cac4e8-723b-4b9f-aeb5-73b3ce35d1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:13.777145 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cac4e8-723b-4b9f-aeb5-73b3ce35d1ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:13.777791 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cac4e8-723b-4b9f-aeb5-73b3ce35d1ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:13.779862 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cac4e8-723b-4b9f-aeb5-73b3ce35d1ce admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:13.793661 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 841/1678] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 14:26:13.848930 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 838/1679] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:26:13 2022] POST /identity/v3/projects => generated 363 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:13.850672 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30136c2f-0759-49b8-92b5-e87e9c5c8cac service neutron] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:13.873619 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30136c2f-0759-49b8-92b5-e87e9c5c8cac service neutron] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:13.956970 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd5c46-5e0a-4d51-bb9f-9368faa61e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:13.958162 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd5c46-5e0a-4d51-bb9f-9368faa61e4e admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:13.958673 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd5c46-5e0a-4d51-bb9f-9368faa61e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:13.959483 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd5c46-5e0a-4d51-bb9f-9368faa61e4e admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:13.960333 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd5c46-5e0a-4d51-bb9f-9368faa61e4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:13.963742 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd5c46-5e0a-4d51-bb9f-9368faa61e4e admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:14.003609 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 842/1680] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:26:13 2022] POST /identity/v3/users => generated 282 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:14.004248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cac4e8-723b-4b9f-aeb5-73b3ce35d1ce admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:14.024393 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cac4e8-723b-4b9f-aeb5-73b3ce35d1ce admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:14.044141 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d613ca6-3d51-4ff0-ad0b-37fbf85f5258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:14.044782 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d613ca6-3d51-4ff0-ad0b-37fbf85f5258 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:14.045146 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d613ca6-3d51-4ff0-ad0b-37fbf85f5258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:14.045312 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d613ca6-3d51-4ff0-ad0b-37fbf85f5258 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:14.046026 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d613ca6-3d51-4ff0-ad0b-37fbf85f5258 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:14.047794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d613ca6-3d51-4ff0-ad0b-37fbf85f5258 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:14.048351 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923d4e7-1a3f-43fb-bb86-54c23691f178 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:14.049038 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923d4e7-1a3f-43fb-bb86-54c23691f178 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:14.050954 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923d4e7-1a3f-43fb-bb86-54c23691f178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:14.060489 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:14.082225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 839/1681] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:14.085092 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d613ca6-3d51-4ff0-ad0b-37fbf85f5258 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:14.107522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d613ca6-3d51-4ff0-ad0b-37fbf85f5258 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:14.123171 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd19c4ea-81c2-4b03-b9c7-1c74367bb018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:14.124465 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19c4ea-81c2-4b03-b9c7-1c74367bb018 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:14.125458 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19c4ea-81c2-4b03-b9c7-1c74367bb018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:14.125458 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19c4ea-81c2-4b03-b9c7-1c74367bb018 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:14.125962 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd19c4ea-81c2-4b03-b9c7-1c74367bb018 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:14.128280 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd19c4ea-81c2-4b03-b9c7-1c74367bb018 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:14.134901 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-f923d4e7-1a3f-43fb-bb86-54c23691f178 None None] MFA Rules not processed for user `75575c10a8eb4ebfab5b2f0c330541df`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:14.136610 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-f923d4e7-1a3f-43fb-bb86-54c23691f178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:14.176177 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 840/1682] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:14.177239 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd19c4ea-81c2-4b03-b9c7-1c74367bb018 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:14.215255 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd19c4ea-81c2-4b03-b9c7-1c74367bb018 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:14.224115 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14fcef-1673-438b-b607-258a01c6622f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:14.224901 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14fcef-1673-438b-b607-258a01c6622f admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:14.225289 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14fcef-1673-438b-b607-258a01c6622f admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:14.225657 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14fcef-1673-438b-b607-258a01c6622f admin admin] PATH_INFO: `/v3/projects/a8c31dbf0edd44c689709c087a51e796/users/a60ca6fcc27e49bdba767217a0e537c7/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:14.243625 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 843/1683] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:14.256934 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c8584-f016-459f-b0fc-1d0306aad090 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:14.257624 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c8584-f016-459f-b0fc-1d0306aad090 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:14.258382 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c8584-f016-459f-b0fc-1d0306aad090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:14.268702 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14fcef-1673-438b-b607-258a01c6622f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8c31dbf0edd44c689709c087a51e796, user_id=a60ca6fcc27e49bdba767217a0e537c7, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:14.276660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14fcef-1673-438b-b607-258a01c6622f admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:14.297715 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-9c1c8584-f016-459f-b0fc-1d0306aad090 None None] MFA Rules not processed for user `75575c10a8eb4ebfab5b2f0c330541df`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:14.302667 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-9c1c8584-f016-459f-b0fc-1d0306aad090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:14.312999 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 841/1684] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:14 2022] PUT /identity/v3/projects/a8c31dbf0edd44c689709c087a51e796/users/a60ca6fcc27e49bdba767217a0e537c7/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:14.325271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef931a0-4a71-490b-aac6-4914dbef022b None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:14.325762 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef931a0-4a71-490b-aac6-4914dbef022b None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:14.326254 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef931a0-4a71-490b-aac6-4914dbef022b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:14.352367 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:14.434521 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 844/1685] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:14.437774 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-2ef931a0-4a71-490b-aac6-4914dbef022b None None] MFA Rules not processed for user `a60ca6fcc27e49bdba767217a0e537c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:14.439181 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-2ef931a0-4a71-490b-aac6-4914dbef022b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:14.442123 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c8584-f016-459f-b0fc-1d0306aad090 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:14.443669 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-9c1c8584-f016-459f-b0fc-1d0306aad090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:14.531039 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 842/1686] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:14.532034 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef931a0-4a71-490b-aac6-4914dbef022b None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:14.625527 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef931a0-4a71-490b-aac6-4914dbef022b None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:14.631650 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a6eb8-6934-4178-8001-26f0ab709e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:14.633307 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a6eb8-6934-4178-8001-26f0ab709e10 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:14.633668 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a6eb8-6934-4178-8001-26f0ab709e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:14.634014 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a6eb8-6934-4178-8001-26f0ab709e10 admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:14.634917 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8a6eb8-6934-4178-8001-26f0ab709e10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:14.636966 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8a6eb8-6934-4178-8001-26f0ab709e10 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:14.658784 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c8584-f016-459f-b0fc-1d0306aad090 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:14.679576 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c75ea2-9ea0-40ba-b241-8e1302037ef4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:14.680315 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c75ea2-9ea0-40ba-b241-8e1302037ef4 service nova] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:14.680806 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c75ea2-9ea0-40ba-b241-8e1302037ef4 service nova] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:14.681291 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c75ea2-9ea0-40ba-b241-8e1302037ef4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:14.688842 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 843/1687] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:26:14 2022] POST /identity/v3/projects => generated 363 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:14.690057 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a6eb8-6934-4178-8001-26f0ab709e10 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:14.718565 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c75ea2-9ea0-40ba-b241-8e1302037ef4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:14.725161 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c75ea2-9ea0-40ba-b241-8e1302037ef4 service nova] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:14.757156 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 845/1688] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:14 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:14.801842 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:14.847846 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a6eb8-6934-4178-8001-26f0ab709e10 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:14.862852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c75ea2-9ea0-40ba-b241-8e1302037ef4 service nova] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:14.866715 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-b0c75ea2-9ea0-40ba-b241-8e1302037ef4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:14.881252 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e834abcb-63b7-4d5f-a3ac-938325af73d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:14.881893 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834abcb-63b7-4d5f-a3ac-938325af73d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:14.888361 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834abcb-63b7-4d5f-a3ac-938325af73d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:14.888830 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834abcb-63b7-4d5f-a3ac-938325af73d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:14.893398 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e834abcb-63b7-4d5f-a3ac-938325af73d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:14.917460 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e834abcb-63b7-4d5f-a3ac-938325af73d5 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:14.940438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c75ea2-9ea0-40ba-b241-8e1302037ef4 service nova] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:14.948009 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98beee0e-69d2-44ab-8918-f755f5f7a15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:14.949344 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98beee0e-69d2-44ab-8918-f755f5f7a15c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:14.950142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98beee0e-69d2-44ab-8918-f755f5f7a15c admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:14.950643 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98beee0e-69d2-44ab-8918-f755f5f7a15c admin admin] PATH_INFO: `/v3/projects/0de544bee8754ed6bd7815538515f1f7` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:14.952168 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98beee0e-69d2-44ab-8918-f755f5f7a15c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0de544bee8754ed6bd7815538515f1f7)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:14.955968 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98beee0e-69d2-44ab-8918-f755f5f7a15c admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:15.001924 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 844/1689] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:26:14 2022] POST /identity/v3/users => generated 288 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:15.007164 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e834abcb-63b7-4d5f-a3ac-938325af73d5 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:15.030917 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e834abcb-63b7-4d5f-a3ac-938325af73d5 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:15.050258 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-98beee0e-69d2-44ab-8918-f755f5f7a15c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0de544bee8754ed6bd7815538515f1f7', 'request_id': 'req-98beee0e-69d2-44ab-8918-f755f5f7a15c'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:15.077367 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-98beee0e-69d2-44ab-8918-f755f5f7a15c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0de544bee8754ed6bd7815538515f1f7', 'request_id': 'req-98beee0e-69d2-44ab-8918-f755f5f7a15c'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:15.096258 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-98beee0e-69d2-44ab-8918-f755f5f7a15c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0de544bee8754ed6bd7815538515f1f7', 'request_id': 'req-98beee0e-69d2-44ab-8918-f755f5f7a15c'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:15.136450 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7f61ba-d057-4b8e-b9ed-b1365f367e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:15.137086 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f61ba-d057-4b8e-b9ed-b1365f367e6a admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:15.137624 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f61ba-d057-4b8e-b9ed-b1365f367e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:15.137908 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f61ba-d057-4b8e-b9ed-b1365f367e6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:15.141576 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-98beee0e-69d2-44ab-8918-f755f5f7a15c admin admin] The token cache is being invalidate because project 0de544bee8754ed6bd7815538515f1f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:26:15.141657 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-98beee0e-69d2-44ab-8918-f755f5f7a15c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:15.141721 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7f61ba-d057-4b8e-b9ed-b1365f367e6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:15.145362 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7f61ba-d057-4b8e-b9ed-b1365f367e6a admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:15.149965 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-98beee0e-69d2-44ab-8918-f755f5f7a15c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:15.159400 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 846/1690] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:26:14 2022] DELETE /identity/v3/projects/0de544bee8754ed6bd7815538515f1f7 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:15.159400 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe3034-f08a-4525-80ea-8e0aec6e4ec7 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:15.159400 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe3034-f08a-4525-80ea-8e0aec6e4ec7 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:15.160442 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe3034-f08a-4525-80ea-8e0aec6e4ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:15.171876 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 845/1691] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:15 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:15.173254 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0e1e1-c328-4bc0-a41b-68f9d9e87089 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:15.174180 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0e1e1-c328-4bc0-a41b-68f9d9e87089 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:15.174711 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0e1e1-c328-4bc0-a41b-68f9d9e87089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:15.212623 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-6eb0e1e1-c328-4bc0-a41b-68f9d9e87089 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:15.214331 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-6eb0e1e1-c328-4bc0-a41b-68f9d9e87089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:15.233665 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 846/1692] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:15.233665 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb0e1e1-c328-4bc0-a41b-68f9d9e87089 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:15.233665 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-6eb0e1e1-c328-4bc0-a41b-68f9d9e87089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:15.236405 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-fdbe3034-f08a-4525-80ea-8e0aec6e4ec7 None None] MFA Rules not processed for user `6c14990386754fd79905a63dd2606eb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:15.237229 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-fdbe3034-f08a-4525-80ea-8e0aec6e4ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:15.258268 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb0e1e1-c328-4bc0-a41b-68f9d9e87089 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:15.267784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca1fa5-e122-421c-89e2-12d2848b8b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:15.268477 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca1fa5-e122-421c-89e2-12d2848b8b17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:15.268755 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca1fa5-e122-421c-89e2-12d2848b8b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:15.269028 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca1fa5-e122-421c-89e2-12d2848b8b17 admin admin] PATH_INFO: `/v3/projects/4253cdafdba847259586722b26822d68/users/6b97e389a3344331a52d9c3c7906329a/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:15.285951 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:15.310859 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ca1fa5-e122-421c-89e2-12d2848b8b17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4253cdafdba847259586722b26822d68, user_id=6b97e389a3344331a52d9c3c7906329a, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:15.318409 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ca1fa5-e122-421c-89e2-12d2848b8b17 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:15.339908 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 847/1693] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:15 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:15.339908 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6244a8-eada-4e6e-a706-291fa7a9179f None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:15.340148 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6244a8-eada-4e6e-a706-291fa7a9179f None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:15.340514 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6244a8-eada-4e6e-a706-291fa7a9179f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:15.355486 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 847/1694] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:15 2022] PUT /identity/v3/projects/4253cdafdba847259586722b26822d68/users/6b97e389a3344331a52d9c3c7906329a/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:15.357452 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca1fa5-e122-421c-89e2-12d2848b8b17 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:15.361830 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-89ca1fa5-e122-421c-89e2-12d2848b8b17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:15.372866 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-0f6244a8-eada-4e6e-a706-291fa7a9179f None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:15.374247 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-0f6244a8-eada-4e6e-a706-291fa7a9179f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:15.510769 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 848/1695] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:15.513787 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6244a8-eada-4e6e-a706-291fa7a9179f None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:15.523573 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca1fa5-e122-421c-89e2-12d2848b8b17 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:15.535033 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a12838-b3cb-4b22-9254-6249f3120761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:15.535670 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a12838-b3cb-4b22-9254-6249f3120761 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:15.535951 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a12838-b3cb-4b22-9254-6249f3120761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:15.536226 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a12838-b3cb-4b22-9254-6249f3120761 admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:15.537915 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a12838-b3cb-4b22-9254-6249f3120761 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:15.538021 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6244a8-eada-4e6e-a706-291fa7a9179f None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:15.541023 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a12838-b3cb-4b22-9254-6249f3120761 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:15.547259 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47de0f5-d86e-4f28-bebf-c1aa94bb0284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:15.547824 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47de0f5-d86e-4f28-bebf-c1aa94bb0284 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:15.548104 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47de0f5-d86e-4f28-bebf-c1aa94bb0284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:15.548373 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47de0f5-d86e-4f28-bebf-c1aa94bb0284 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:15.549020 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47de0f5-d86e-4f28-bebf-c1aa94bb0284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:15.551143 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47de0f5-d86e-4f28-bebf-c1aa94bb0284 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:15.579650 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 849/1696] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:15 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:15.584052 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47de0f5-d86e-4f28-bebf-c1aa94bb0284 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:15.604713 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 848/1697] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:26:15 2022] POST /identity/v3/projects => generated 405 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:15.605814 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a12838-b3cb-4b22-9254-6249f3120761 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:15.623971 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47de0f5-d86e-4f28-bebf-c1aa94bb0284 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:15.670037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:15.763556 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db4f0b6-8904-4f36-bef1-37d7bbf8f94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:15.764584 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4f0b6-8904-4f36-bef1-37d7bbf8f94f admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:15.765037 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4f0b6-8904-4f36-bef1-37d7bbf8f94f admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:15.765476 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4f0b6-8904-4f36-bef1-37d7bbf8f94f admin admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:15.769156 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db4f0b6-8904-4f36-bef1-37d7bbf8f94f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:26:15.769716 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db4f0b6-8904-4f36-bef1-37d7bbf8f94f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:15.773908 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db4f0b6-8904-4f36-bef1-37d7bbf8f94f admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:15.799750 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 850/1698] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:26:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:26:15.801043 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db4f0b6-8904-4f36-bef1-37d7bbf8f94f admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:15.807880 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a12838-b3cb-4b22-9254-6249f3120761 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:15.826603 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbda7d76-ae66-44c3-a629-d44276a7a4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:15.827067 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db4f0b6-8904-4f36-bef1-37d7bbf8f94f admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:15.827402 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda7d76-ae66-44c3-a629-d44276a7a4ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:15.827684 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda7d76-ae66-44c3-a629-d44276a7a4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:15.828129 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda7d76-ae66-44c3-a629-d44276a7a4ea admin admin] PATH_INFO: `/v3/projects/4253cdafdba847259586722b26822d68/users/6b97e389a3344331a52d9c3c7906329a/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:15.833185 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbda7d76-ae66-44c3-a629-d44276a7a4ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=4253cdafdba847259586722b26822d68, user_id=6b97e389a3344331a52d9c3c7906329a, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:15.845941 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e142e9c-75fc-403a-ac52-e17440a1d8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:15.846850 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e142e9c-75fc-403a-ac52-e17440a1d8c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:15.847321 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e142e9c-75fc-403a-ac52-e17440a1d8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:15.848062 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbda7d76-ae66-44c3-a629-d44276a7a4ea admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:15.853817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e142e9c-75fc-403a-ac52-e17440a1d8c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:15.854852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e142e9c-75fc-403a-ac52-e17440a1d8c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:15.860485 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e142e9c-75fc-403a-ac52-e17440a1d8c1 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:15.874840 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 849/1699] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:15 2022] PUT /identity/v3/projects/4253cdafdba847259586722b26822d68/users/6b97e389a3344331a52d9c3c7906329a/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:15.874840 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbda7d76-ae66-44c3-a629-d44276a7a4ea admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:15.914236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 851/1700] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:26:15 2022] POST /identity/v3/users => generated 309 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:15.915797 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a0194-cd11-438f-9e84-013143a4550c None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:15.916084 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a0194-cd11-438f-9e84-013143a4550c None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:15.916401 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a0194-cd11-438f-9e84-013143a4550c None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:15.923775 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 852/1701] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 14:26:15 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:26:15.925185 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbcf4ec-a3b8-4124-9caf-f1ad06436973 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:15.925533 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbcf4ec-a3b8-4124-9caf-f1ad06436973 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:15.925859 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbcf4ec-a3b8-4124-9caf-f1ad06436973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:15.967789 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-4dbcf4ec-a3b8-4124-9caf-f1ad06436973 None None] MFA Rules not processed for user `6b97e389a3344331a52d9c3c7906329a`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:15.969319 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-4dbcf4ec-a3b8-4124-9caf-f1ad06436973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:16.022513 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:16.035253 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbda7d76-ae66-44c3-a629-d44276a7a4ea admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:16.057654 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a89067-0355-4135-88e7-954563a84292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:16.058624 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a89067-0355-4135-88e7-954563a84292 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:16.059393 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a89067-0355-4135-88e7-954563a84292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:16.059672 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a89067-0355-4135-88e7-954563a84292 admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:16.060779 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a89067-0355-4135-88e7-954563a84292 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:16.071234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a89067-0355-4135-88e7-954563a84292 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:16.115981 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 853/1702] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:15 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:16.118244 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbcf4ec-a3b8-4124-9caf-f1ad06436973 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:16.135104 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbcf4ec-a3b8-4124-9caf-f1ad06436973 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:16.140600 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca2419-d853-407d-9ea8-fe62d4c407ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:16.141610 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca2419-d853-407d-9ea8-fe62d4c407ad admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:16.142097 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca2419-d853-407d-9ea8-fe62d4c407ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:16.142562 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca2419-d853-407d-9ea8-fe62d4c407ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:16.143766 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 850/1703] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:26:15 2022] POST /identity/v3/projects => generated 369 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:16.144378 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deca2419-d853-407d-9ea8-fe62d4c407ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:16.146241 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deca2419-d853-407d-9ea8-fe62d4c407ad admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:16.147232 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cbe172-7697-440f-af76-a7e438054e49 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:16.147539 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cbe172-7697-440f-af76-a7e438054e49 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:16.147884 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cbe172-7697-440f-af76-a7e438054e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:16.165334 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 854/1704] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:16 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:16.166259 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca2419-d853-407d-9ea8-fe62d4c407ad admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:16.173766 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-deca2419-d853-407d-9ea8-fe62d4c407ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:16.243472 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-73cbe172-7697-440f-af76-a7e438054e49 None None] MFA Rules not processed for user `a9348164132846d2b593ddfa29a820eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:16.245965 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-73cbe172-7697-440f-af76-a7e438054e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:16.324898 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca2419-d853-407d-9ea8-fe62d4c407ad admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:16.361750 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 851/1705] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 14:26:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:16.369470 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cbe172-7697-440f-af76-a7e438054e49 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:16.432270 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:16.478095 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cbe172-7697-440f-af76-a7e438054e49 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:16.484456 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa2fbf8-18ed-4269-b412-5178176b67bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:16.485153 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d8d93-f916-4226-8b08-2eca4e323b8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:16.485581 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2fbf8-18ed-4269-b412-5178176b67bf admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:16.485872 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2fbf8-18ed-4269-b412-5178176b67bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:16.486148 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2fbf8-18ed-4269-b412-5178176b67bf admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:16.486401 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d8d93-f916-4226-8b08-2eca4e323b8e service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:16.486795 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa2fbf8-18ed-4269-b412-5178176b67bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:16.486972 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d8d93-f916-4226-8b08-2eca4e323b8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:16.487376 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d8d93-f916-4226-8b08-2eca4e323b8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:16.488768 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa2fbf8-18ed-4269-b412-5178176b67bf admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:16.526046 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 852/1706] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:26:16 2022] POST /identity/v3/users => generated 285 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:16.531798 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa2fbf8-18ed-4269-b412-5178176b67bf admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:16.556678 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa2fbf8-18ed-4269-b412-5178176b67bf admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:16.570252 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c521e-acdb-4676-8a99-4237ac2eccd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:16.570788 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c521e-acdb-4676-8a99-4237ac2eccd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:16.571101 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c521e-acdb-4676-8a99-4237ac2eccd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:16.571481 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c521e-acdb-4676-8a99-4237ac2eccd0 admin admin] PATH_INFO: `/v3/projects/5f17177494944cfb9602cd48a91a2d34/users/a9bd5d645cd444e7ba404060de7f0d7f/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:16.598519 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387d8d93-f916-4226-8b08-2eca4e323b8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:16.598519 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387d8d93-f916-4226-8b08-2eca4e323b8e service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:16.603974 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5c521e-acdb-4676-8a99-4237ac2eccd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f17177494944cfb9602cd48a91a2d34, user_id=a9bd5d645cd444e7ba404060de7f0d7f, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:16.606671 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5c521e-acdb-4676-8a99-4237ac2eccd0 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:16.629870 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 855/1707] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:16 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:16.630778 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d8d93-f916-4226-8b08-2eca4e323b8e service cinder] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:16.635168 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-387d8d93-f916-4226-8b08-2eca4e323b8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:16.639119 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 853/1708] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:16 2022] PUT /identity/v3/projects/5f17177494944cfb9602cd48a91a2d34/users/a9bd5d645cd444e7ba404060de7f0d7f/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:16.639719 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c521e-acdb-4676-8a99-4237ac2eccd0 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:16.660394 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d8d93-f916-4226-8b08-2eca4e323b8e service cinder] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:16.688527 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:16.761142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a110a2d-47c4-4e64-b0d5-37df89324554 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:16.761918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a110a2d-47c4-4e64-b0d5-37df89324554 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:16.762032 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a110a2d-47c4-4e64-b0d5-37df89324554 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:16.762325 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a110a2d-47c4-4e64-b0d5-37df89324554 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:16.768261 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c521e-acdb-4676-8a99-4237ac2eccd0 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:16.775610 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a110a2d-47c4-4e64-b0d5-37df89324554 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:16.777696 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a110a2d-47c4-4e64-b0d5-37df89324554 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:16.783901 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6977aade-282e-4285-891f-156d7d0c8717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:16.784538 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977aade-282e-4285-891f-156d7d0c8717 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:16.784817 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977aade-282e-4285-891f-156d7d0c8717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:16.785103 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977aade-282e-4285-891f-156d7d0c8717 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:16.785755 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6977aade-282e-4285-891f-156d7d0c8717 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:16.787554 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6977aade-282e-4285-891f-156d7d0c8717 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:16.798910 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 856/1709] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:16.799787 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a110a2d-47c4-4e64-b0d5-37df89324554 service neutron] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:16.808325 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 854/1710] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:16 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:16.809046 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6977aade-282e-4285-891f-156d7d0c8717 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:16.824610 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a110a2d-47c4-4e64-b0d5-37df89324554 service neutron] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:16.844713 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cde8e5-c149-43bc-87a2-e0f7504a951d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:16.845364 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cde8e5-c149-43bc-87a2-e0f7504a951d admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:16.845647 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cde8e5-c149-43bc-87a2-e0f7504a951d admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:16.846730 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cde8e5-c149-43bc-87a2-e0f7504a951d admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:16.848455 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cde8e5-c149-43bc-87a2-e0f7504a951d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:16.852763 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cde8e5-c149-43bc-87a2-e0f7504a951d admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:16.888529 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 857/1711] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:16 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:16.888529 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cde8e5-c149-43bc-87a2-e0f7504a951d admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:16.922400 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cde8e5-c149-43bc-87a2-e0f7504a951d admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:16.934447 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ead142-04a3-4a1d-a39c-64329a77d58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:16.938862 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ead142-04a3-4a1d-a39c-64329a77d58f admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:16.938862 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ead142-04a3-4a1d-a39c-64329a77d58f admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:16.939184 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ead142-04a3-4a1d-a39c-64329a77d58f admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:16.940926 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ead142-04a3-4a1d-a39c-64329a77d58f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:16.946039 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ead142-04a3-4a1d-a39c-64329a77d58f admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:16.951220 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6977aade-282e-4285-891f-156d7d0c8717 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:16.957342 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900fd0ae-d18a-4670-851a-0f1b58eb0673 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:16.958748 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900fd0ae-d18a-4670-851a-0f1b58eb0673 service cinder] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:16.959531 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900fd0ae-d18a-4670-851a-0f1b58eb0673 service cinder] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:16.959964 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900fd0ae-d18a-4670-851a-0f1b58eb0673 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:16.966446 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-900fd0ae-d18a-4670-851a-0f1b58eb0673 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:16.991816 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 858/1712] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:16 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:16.997147 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ead142-04a3-4a1d-a39c-64329a77d58f admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:17.035924 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ead142-04a3-4a1d-a39c-64329a77d58f admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:17.050773 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d94d4f-9440-4940-a7b9-67746f32dc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:17.066399 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d94d4f-9440-4940-a7b9-67746f32dc81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:17.067432 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d94d4f-9440-4940-a7b9-67746f32dc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:17.068238 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d94d4f-9440-4940-a7b9-67746f32dc81 admin admin] PATH_INFO: `/v3/projects/5f17177494944cfb9602cd48a91a2d34/users/a9bd5d645cd444e7ba404060de7f0d7f/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:17.074177 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d94d4f-9440-4940-a7b9-67746f32dc81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f17177494944cfb9602cd48a91a2d34, user_id=a9bd5d645cd444e7ba404060de7f0d7f, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:17.084050 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d94d4f-9440-4940-a7b9-67746f32dc81 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:17.124307 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:17.152373 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 859/1713] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:16 2022] PUT /identity/v3/projects/5f17177494944cfb9602cd48a91a2d34/users/a9bd5d645cd444e7ba404060de7f0d7f/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:17.153689 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d94d4f-9440-4940-a7b9-67746f32dc81 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:17.168700 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900fd0ae-d18a-4670-851a-0f1b58eb0673 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:17.174733 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900fd0ae-d18a-4670-851a-0f1b58eb0673 service cinder] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:17.218620 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 855/1714] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:16 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:17.317463 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d94d4f-9440-4940-a7b9-67746f32dc81 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:17.331862 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d91f5-1ac8-4579-940d-fc83acafa149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:17.333382 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d91f5-1ac8-4579-940d-fc83acafa149 admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:17.333840 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d91f5-1ac8-4579-940d-fc83acafa149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:17.334800 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d91f5-1ac8-4579-940d-fc83acafa149 admin admin] PATH_INFO: `/v3/projects/39ed972dbf42400988bb9cdebfad21f5/users/e90b779cac3d4ab6a2ad37582fc091fa/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:17.375620 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d91f5-1ac8-4579-940d-fc83acafa149 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39ed972dbf42400988bb9cdebfad21f5, user_id=e90b779cac3d4ab6a2ad37582fc091fa, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:17.378727 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d91f5-1ac8-4579-940d-fc83acafa149 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:17.410955 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 860/1715] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:17 2022] PUT /identity/v3/projects/39ed972dbf42400988bb9cdebfad21f5/users/e90b779cac3d4ab6a2ad37582fc091fa/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:17.417121 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a2041-c4ae-46d7-9671-1fa07a86a2d6 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:17.418653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a2041-c4ae-46d7-9671-1fa07a86a2d6 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:17.419096 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a2041-c4ae-46d7-9671-1fa07a86a2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:17.479479 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-045a2041-c4ae-46d7-9671-1fa07a86a2d6 None None] MFA Rules not processed for user `e90b779cac3d4ab6a2ad37582fc091fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:17.480708 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-045a2041-c4ae-46d7-9671-1fa07a86a2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:17.629224 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 856/1716] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:17 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:17.634865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d91f5-1ac8-4579-940d-fc83acafa149 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:17.785433 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d91f5-1ac8-4579-940d-fc83acafa149 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:17.812150 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da0b94-50a0-460e-a03e-8ad682bdc9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:17.813466 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da0b94-50a0-460e-a03e-8ad682bdc9a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:17.814883 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da0b94-50a0-460e-a03e-8ad682bdc9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:17.815647 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da0b94-50a0-460e-a03e-8ad682bdc9a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:17.820644 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88da0b94-50a0-460e-a03e-8ad682bdc9a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:17.820644 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88da0b94-50a0-460e-a03e-8ad682bdc9a2 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:17.851522 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:17.877451 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a2041-c4ae-46d7-9671-1fa07a86a2d6 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:17.880099 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-045a2041-c4ae-46d7-9671-1fa07a86a2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:17.897401 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 861/1717] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:26:17 2022] POST /identity/v3/projects => generated 369 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:17.899813 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da0b94-50a0-460e-a03e-8ad682bdc9a2 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:18.087130 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da0b94-50a0-460e-a03e-8ad682bdc9a2 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:18.102312 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a2041-c4ae-46d7-9671-1fa07a86a2d6 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:18.104944 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b17d96e-9e52-4b65-8e14-b731a5804e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:18.105583 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b17d96e-9e52-4b65-8e14-b731a5804e58 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:18.106227 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b17d96e-9e52-4b65-8e14-b731a5804e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:18.106227 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b17d96e-9e52-4b65-8e14-b731a5804e58 admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:18.106767 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b17d96e-9e52-4b65-8e14-b731a5804e58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:18.110798 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b17d96e-9e52-4b65-8e14-b731a5804e58 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:18.166340 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 862/1718] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:26:17 2022] POST /identity/v3/users => generated 291 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:18.169175 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b17d96e-9e52-4b65-8e14-b731a5804e58 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:18.190503 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b17d96e-9e52-4b65-8e14-b731a5804e58 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:18.204191 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e135d3-7ec6-4538-81fb-8fc2a13127e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:18.205235 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e135d3-7ec6-4538-81fb-8fc2a13127e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:18.205514 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e135d3-7ec6-4538-81fb-8fc2a13127e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:18.205785 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e135d3-7ec6-4538-81fb-8fc2a13127e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:18.208604 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e135d3-7ec6-4538-81fb-8fc2a13127e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:18.213411 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e135d3-7ec6-4538-81fb-8fc2a13127e2 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:18.244035 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 863/1719] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:18 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:18.245296 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e135d3-7ec6-4538-81fb-8fc2a13127e2 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:18.265416 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e135d3-7ec6-4538-81fb-8fc2a13127e2 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:18.278464 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a76cb3-9461-431b-94d9-38ba57294d59 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18fe384ecd454fb298dcc788f41dd44b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:18.279518 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a76cb3-9461-431b-94d9-38ba57294d59 service placement] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:18.280016 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a76cb3-9461-431b-94d9-38ba57294d59 service placement] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:18.280658 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a76cb3-9461-431b-94d9-38ba57294d59 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:18.286291 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-29a76cb3-9461-431b-94d9-38ba57294d59 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:18.297570 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a87bc68-fed7-4167-8456-e2cd6f303716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:18.298262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87bc68-fed7-4167-8456-e2cd6f303716 admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:18.298543 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87bc68-fed7-4167-8456-e2cd6f303716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:18.298835 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87bc68-fed7-4167-8456-e2cd6f303716 admin admin] PATH_INFO: `/v3/projects/ef05a1a524df41b4adbd8f0fcde5725a/users/c8ee73f10f174d1c95824575df1eb3a3/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:18.358399 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a76cb3-9461-431b-94d9-38ba57294d59 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:18.360439 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a76cb3-9461-431b-94d9-38ba57294d59 service placement] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:18.390914 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a87bc68-fed7-4167-8456-e2cd6f303716 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef05a1a524df41b4adbd8f0fcde5725a, user_id=c8ee73f10f174d1c95824575df1eb3a3, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:18.399306 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a87bc68-fed7-4167-8456-e2cd6f303716 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:18.408073 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 857/1720] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:18.440970 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 864/1721] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:18 2022] PUT /identity/v3/projects/ef05a1a524df41b4adbd8f0fcde5725a/users/c8ee73f10f174d1c95824575df1eb3a3/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:18.447302 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a76cb3-9461-431b-94d9-38ba57294d59 service placement] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:18.481411 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a76cb3-9461-431b-94d9-38ba57294d59 service placement] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:18.503593 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:18.668252 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b1099a-a197-4aa1-87c8-c1e13b42be53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:18.668891 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1099a-a197-4aa1-87c8-c1e13b42be53 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:18.669171 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1099a-a197-4aa1-87c8-c1e13b42be53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:18.669445 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1099a-a197-4aa1-87c8-c1e13b42be53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:18.670477 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b1099a-a197-4aa1-87c8-c1e13b42be53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:18.674852 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b1099a-a197-4aa1-87c8-c1e13b42be53 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:18.706825 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 858/1722] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:18 2022] GET /identity/v3/roles => generated 1420 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:18.710495 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a87bc68-fed7-4167-8456-e2cd6f303716 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:18.723056 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a87bc68-fed7-4167-8456-e2cd6f303716 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:18.752631 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a388943f-9f57-462f-9690-826f0bd9e766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:18.753327 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388943f-9f57-462f-9690-826f0bd9e766 admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:18.753610 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388943f-9f57-462f-9690-826f0bd9e766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:18.753881 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388943f-9f57-462f-9690-826f0bd9e766 admin admin] PATH_INFO: `/v3/projects/ef05a1a524df41b4adbd8f0fcde5725a/users/c8ee73f10f174d1c95824575df1eb3a3/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:18.773439 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a388943f-9f57-462f-9690-826f0bd9e766 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef05a1a524df41b4adbd8f0fcde5725a, user_id=c8ee73f10f174d1c95824575df1eb3a3, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:18.782340 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a388943f-9f57-462f-9690-826f0bd9e766 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:18.826802 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 865/1723] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:18 2022] PUT /identity/v3/projects/ef05a1a524df41b4adbd8f0fcde5725a/users/c8ee73f10f174d1c95824575df1eb3a3/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:18.837456 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd34a90-9e28-4104-8a50-bf3e18ab4a62 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:18.837801 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd34a90-9e28-4104-8a50-bf3e18ab4a62 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:18.838089 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd34a90-9e28-4104-8a50-bf3e18ab4a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:18.934355 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-1fd34a90-9e28-4104-8a50-bf3e18ab4a62 None None] MFA Rules not processed for user `c8ee73f10f174d1c95824575df1eb3a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:18.949245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-1fd34a90-9e28-4104-8a50-bf3e18ab4a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:19.020503 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911bf569-3e25-4b25-8af7-e8d7aefb6925 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:19.020789 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911bf569-3e25-4b25-8af7-e8d7aefb6925 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:19.021051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911bf569-3e25-4b25-8af7-e8d7aefb6925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:19.142116 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-911bf569-3e25-4b25-8af7-e8d7aefb6925 None None] MFA Rules not processed for user `a9bd5d645cd444e7ba404060de7f0d7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:19.148718 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-911bf569-3e25-4b25-8af7-e8d7aefb6925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:19.250092 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 859/1724] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:18 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 414 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:19.263115 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd34a90-9e28-4104-8a50-bf3e18ab4a62 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:19.365618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 866/1725] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:19 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:19.397325 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd34a90-9e28-4104-8a50-bf3e18ab4a62 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:19.405540 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b56288-1875-4a18-bb6e-566a2605c77e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:19.407563 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b56288-1875-4a18-bb6e-566a2605c77e service cinder] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:19.407854 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b56288-1875-4a18-bb6e-566a2605c77e service cinder] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:19.408235 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b56288-1875-4a18-bb6e-566a2605c77e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:19.438248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b56288-1875-4a18-bb6e-566a2605c77e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:19.440989 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b56288-1875-4a18-bb6e-566a2605c77e service cinder] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:19.472794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 860/1726] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:19 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:19.552524 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3566d-1d9b-450a-aabc-6258dbdbced7 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:19.552983 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3566d-1d9b-450a-aabc-6258dbdbced7 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:19.553051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3566d-1d9b-450a-aabc-6258dbdbced7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:19.714400 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-2df3566d-1d9b-450a-aabc-6258dbdbced7 None None] MFA Rules not processed for user `c3cf8e31062d4bd7bfa76c09e7e2adea`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:19.719757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-2df3566d-1d9b-450a-aabc-6258dbdbced7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:19.742671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:19.933716 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 867/1727] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:19 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:19.945571 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012e9f4-d3dd-407a-bc63-2fe3c8b481ae None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:19.945946 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012e9f4-d3dd-407a-bc63-2fe3c8b481ae None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:19.946393 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012e9f4-d3dd-407a-bc63-2fe3c8b481ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:19.978776 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-0012e9f4-d3dd-407a-bc63-2fe3c8b481ae None None] MFA Rules not processed for user `c3cf8e31062d4bd7bfa76c09e7e2adea`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:19.980389 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-0012e9f4-d3dd-407a-bc63-2fe3c8b481ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:19.991384 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 861/1728] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:19 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:20.002663 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df3566d-1d9b-450a-aabc-6258dbdbced7 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:20.004940 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-2df3566d-1d9b-450a-aabc-6258dbdbced7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:20.205500 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df3566d-1d9b-450a-aabc-6258dbdbced7 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:20.221789 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4a1b4-f2d2-44d8-8a3c-16b09c57e308 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:20.224508 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4a1b4-f2d2-44d8-8a3c-16b09c57e308 service nova] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:20.224988 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4a1b4-f2d2-44d8-8a3c-16b09c57e308 service nova] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:20.227446 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4a1b4-f2d2-44d8-8a3c-16b09c57e308 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:20.249607 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be4a1b4-f2d2-44d8-8a3c-16b09c57e308 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:20.252424 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be4a1b4-f2d2-44d8-8a3c-16b09c57e308 service nova] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:20.295265 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 868/1729] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:20 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:21.502574 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a552ce-1dac-4c66-9bdc-452d2853af39 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:21.503443 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a552ce-1dac-4c66-9bdc-452d2853af39 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:21.503859 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a552ce-1dac-4c66-9bdc-452d2853af39 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:21.505391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 862/1730] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 14:26:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:26:21.532315 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff27ae5-a21a-4e0e-80ed-c36c79e3fc5e None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:21.533719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff27ae5-a21a-4e0e-80ed-c36c79e3fc5e None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:21.534141 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff27ae5-a21a-4e0e-80ed-c36c79e3fc5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:21.569682 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-5ff27ae5-a21a-4e0e-80ed-c36c79e3fc5e None None] MFA Rules not processed for user `a9348164132846d2b593ddfa29a820eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:21.579192 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-5ff27ae5-a21a-4e0e-80ed-c36c79e3fc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:21.685951 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 869/1731] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 14:26:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:21.734014 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a552ce-1dac-4c66-9bdc-452d2853af39 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:21.736365 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-56a552ce-1dac-4c66-9bdc-452d2853af39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:21.771356 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a552ce-1dac-4c66-9bdc-452d2853af39 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:21.786783 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1349a9f8-6535-453e-9f17-d66749dbd1ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:21.787433 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1349a9f8-6535-453e-9f17-d66749dbd1ab service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:21.787716 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1349a9f8-6535-453e-9f17-d66749dbd1ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:21.787990 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1349a9f8-6535-453e-9f17-d66749dbd1ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:21.823068 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1349a9f8-6535-453e-9f17-d66749dbd1ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:21.824592 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1349a9f8-6535-453e-9f17-d66749dbd1ab service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:21.865057 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 863/1732] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:22.437160 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e261c61-bb0c-43b8-8a7c-d5182f712f8f None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:22.437641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e261c61-bb0c-43b8-8a7c-d5182f712f8f None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:22.438077 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e261c61-bb0c-43b8-8a7c-d5182f712f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:22.512025 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-7e261c61-bb0c-43b8-8a7c-d5182f712f8f None None] MFA Rules not processed for user `883e6570afb84a099e46cdace5a88b2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:22.514439 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-7e261c61-bb0c-43b8-8a7c-d5182f712f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:22.618382 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:22.627849 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48629f73-8a72-4daf-8740-c63137cc8f03 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:22.628724 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48629f73-8a72-4daf-8740-c63137cc8f03 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:22.629154 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48629f73-8a72-4daf-8740-c63137cc8f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:22.719281 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 870/1733] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:22 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:22.735093 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e261c61-bb0c-43b8-8a7c-d5182f712f8f None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:22.736669 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-7e261c61-bb0c-43b8-8a7c-d5182f712f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:22.751955 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:22.774563 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e261c61-bb0c-43b8-8a7c-d5182f712f8f None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:22.788735 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3858809d-192f-4e39-b14b-b0c2e1efbc92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:22.789371 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858809d-192f-4e39-b14b-b0c2e1efbc92 service nova] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:22.789654 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858809d-192f-4e39-b14b-b0c2e1efbc92 service nova] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:22.789927 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858809d-192f-4e39-b14b-b0c2e1efbc92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:22.820604 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3858809d-192f-4e39-b14b-b0c2e1efbc92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:22.822401 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3858809d-192f-4e39-b14b-b0c2e1efbc92 service nova] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:22.856635 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-48629f73-8a72-4daf-8740-c63137cc8f03 None None] MFA Rules not processed for user `82328b7fd50649a485c6eaf27c5597c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:22.857851 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-48629f73-8a72-4daf-8740-c63137cc8f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:22.859296 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 871/1734] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:22 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:23.142178 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90042d-0ae2-430a-a508-a41ba6ba8664 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:23.142463 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90042d-0ae2-430a-a508-a41ba6ba8664 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:23.142954 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90042d-0ae2-430a-a508-a41ba6ba8664 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:23.144308 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 872/1735] 10.222.0.52 () {54 vars in 921 bytes} [Sat Sep 24 14:26:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:26:23.174244 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf4a5e-8a3d-4541-8039-d9d2d1482121 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:23.174542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf4a5e-8a3d-4541-8039-d9d2d1482121 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:23.174863 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf4a5e-8a3d-4541-8039-d9d2d1482121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:23.180960 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 864/1736] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 14:26:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 554 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:26:23.237872 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-d2cf4a5e-8a3d-4541-8039-d9d2d1482121 None None] MFA Rules not processed for user `c3cf8e31062d4bd7bfa76c09e7e2adea`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:23.241354 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-d2cf4a5e-8a3d-4541-8039-d9d2d1482121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:23.284506 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 873/1737] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:23 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:26:23.424006 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef3230-c2ed-4f90-8a71-859cfd1d8c5d None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:23.424481 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef3230-c2ed-4f90-8a71-859cfd1d8c5d None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:23.424927 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef3230-c2ed-4f90-8a71-859cfd1d8c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:23.467215 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-20ef3230-c2ed-4f90-8a71-859cfd1d8c5d None None] MFA Rules not processed for user `18fe384ecd454fb298dcc788f41dd44b`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:23.468484 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-20ef3230-c2ed-4f90-8a71-859cfd1d8c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:23.589200 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf4a5e-8a3d-4541-8039-d9d2d1482121 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:23.591022 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-d2cf4a5e-8a3d-4541-8039-d9d2d1482121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:23.639945 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 865/1738] 10.222.0.52 () {58 vars in 1018 bytes} [Sat Sep 24 14:26:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:23.732199 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe841e-205e-4ff3-8aec-546198c8b478 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:23.732598 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe841e-205e-4ff3-8aec-546198c8b478 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:23.733007 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe841e-205e-4ff3-8aec-546198c8b478 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:23.734479 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 866/1739] 10.222.0.52 () {54 vars in 961 bytes} [Sat Sep 24 14:26:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:26:23.742481 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37dec9-e996-406e-82e7-57e178a5fb99 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:23.743377 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37dec9-e996-406e-82e7-57e178a5fb99 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:23.744288 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37dec9-e996-406e-82e7-57e178a5fb99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:23.815795 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-1e37dec9-e996-406e-82e7-57e178a5fb99 None None] MFA Rules not processed for user `18fe384ecd454fb298dcc788f41dd44b`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:23.822995 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-1e37dec9-e996-406e-82e7-57e178a5fb99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:23.873671 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 867/1740] 10.222.0.52 () {58 vars in 1058 bytes} [Sat Sep 24 14:26:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:23.883672 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e37dec9-e996-406e-82e7-57e178a5fb99 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:23.945018 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e37dec9-e996-406e-82e7-57e178a5fb99 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:23.965729 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf4a5e-8a3d-4541-8039-d9d2d1482121 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:23.978388 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2517c-7c62-4921-9498-e0b587b0fbf9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18fe384ecd454fb298dcc788f41dd44b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:23.981812 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e5c46f-2cb7-4987-9146-a631c9eadf35 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '879f296733114f6f844a98aaefe35e92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:23.982489 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2517c-7c62-4921-9498-e0b587b0fbf9 service placement] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:23.982928 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e5c46f-2cb7-4987-9146-a631c9eadf35 service swift] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:23.983995 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2517c-7c62-4921-9498-e0b587b0fbf9 service placement] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:23.984343 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2517c-7c62-4921-9498-e0b587b0fbf9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:23.985581 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e5c46f-2cb7-4987-9146-a631c9eadf35 service swift] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:23.985879 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e5c46f-2cb7-4987-9146-a631c9eadf35 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:24.001821 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b2517c-7c62-4921-9498-e0b587b0fbf9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:24.004745 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b2517c-7c62-4921-9498-e0b587b0fbf9 service placement] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:24.021402 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e5c46f-2cb7-4987-9146-a631c9eadf35 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:24.025630 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e5c46f-2cb7-4987-9146-a631c9eadf35 service swift] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:24.033818 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 868/1741] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:24.060080 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 874/1742] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 14:26:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 14:26:24.857612 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2517c-7c62-4921-9498-e0b587b0fbf9 service placement] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:24.997170 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2517c-7c62-4921-9498-e0b587b0fbf9 service placement] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:25.029472 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf8bf5-78b9-4bf1-9f1f-c4d0d949ffbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:25.030139 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf8bf5-78b9-4bf1-9f1f-c4d0d949ffbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:25.030422 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf8bf5-78b9-4bf1-9f1f-c4d0d949ffbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:25.030699 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf8bf5-78b9-4bf1-9f1f-c4d0d949ffbf admin admin] PATH_INFO: `/v3/users/e90b779cac3d4ab6a2ad37582fc091fa` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:25.032319 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcf8bf5-78b9-4bf1-9f1f-c4d0d949ffbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=e90b779cac3d4ab6a2ad37582fc091fa)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:25.034621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcf8bf5-78b9-4bf1-9f1f-c4d0d949ffbf admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:25.068066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:25.222653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-cfcf8bf5-78b9-4bf1-9f1f-c4d0d949ffbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e90b779cac3d4ab6a2ad37582fc091fa'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:25.225102 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-cfcf8bf5-78b9-4bf1-9f1f-c4d0d949ffbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e90b779cac3d4ab6a2ad37582fc091fa'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:25.235197 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-cfcf8bf5-78b9-4bf1-9f1f-c4d0d949ffbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e90b779cac3d4ab6a2ad37582fc091fa'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:25.236663 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-cfcf8bf5-78b9-4bf1-9f1f-c4d0d949ffbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e90b779cac3d4ab6a2ad37582fc091fa'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:25.272368 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-cfcf8bf5-78b9-4bf1-9f1f-c4d0d949ffbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e90b779cac3d4ab6a2ad37582fc091fa'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:25.314178 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 869/1743] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:26:24 2022] DELETE /identity/v3/users/e90b779cac3d4ab6a2ad37582fc091fa => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:25.318626 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b15851-2e6e-40e2-b3a2-cddeca3d5f1d None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:25.320768 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b15851-2e6e-40e2-b3a2-cddeca3d5f1d None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:25.321251 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b15851-2e6e-40e2-b3a2-cddeca3d5f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:25.378770 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-24b15851-2e6e-40e2-b3a2-cddeca3d5f1d None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:25.380330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-24b15851-2e6e-40e2-b3a2-cddeca3d5f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:25.533922 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 875/1744] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:25.966035 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf8bf5-78b9-4bf1-9f1f-c4d0d949ffbf admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:25.968047 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-cfcf8bf5-78b9-4bf1-9f1f-c4d0d949ffbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:26.143885 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb382c-6be6-4bd5-aadf-158424dfb786 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:26.144177 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb382c-6be6-4bd5-aadf-158424dfb786 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:26.144449 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb382c-6be6-4bd5-aadf-158424dfb786 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:26.145992 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 876/1745] 10.222.0.52 () {54 vars in 921 bytes} [Sat Sep 24 14:26:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:26:26.151659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf8bf5-78b9-4bf1-9f1f-c4d0d949ffbf admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:26.165561 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ec993-5b0f-4bb6-9b08-d9c55dc606c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:26.166433 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ec993-5b0f-4bb6-9b08-d9c55dc606c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:26.166899 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ec993-5b0f-4bb6-9b08-d9c55dc606c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:26.168021 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ec993-5b0f-4bb6-9b08-d9c55dc606c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:26.190964 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c54835-9509-4adc-ae3d-a518f201c387 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:26.191354 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c54835-9509-4adc-ae3d-a518f201c387 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:26.191658 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c54835-9509-4adc-ae3d-a518f201c387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:26.196380 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61ec993-5b0f-4bb6-9b08-d9c55dc606c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:26.200059 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61ec993-5b0f-4bb6-9b08-d9c55dc606c1 service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:26.228786 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 870/1746] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:26.231901 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-94c54835-9509-4adc-ae3d-a518f201c387 None None] MFA Rules not processed for user `18fe384ecd454fb298dcc788f41dd44b`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:26.233135 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-94c54835-9509-4adc-ae3d-a518f201c387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:26.299979 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:26.362790 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ec993-5b0f-4bb6-9b08-d9c55dc606c1 service neutron] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:26.364816 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-f61ec993-5b0f-4bb6-9b08-d9c55dc606c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:26.391971 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ec993-5b0f-4bb6-9b08-d9c55dc606c1 service neutron] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:26.424698 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecba004-d69f-40b3-ae4c-6b3096e9184d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:26.426965 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecba004-d69f-40b3-ae4c-6b3096e9184d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:26.428122 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecba004-d69f-40b3-ae4c-6b3096e9184d admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:26.429895 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecba004-d69f-40b3-ae4c-6b3096e9184d admin admin] PATH_INFO: `/v3/users/6c14990386754fd79905a63dd2606eb0` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:26.432486 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecba004-d69f-40b3-ae4c-6b3096e9184d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c14990386754fd79905a63dd2606eb0)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:26.436200 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecba004-d69f-40b3-ae4c-6b3096e9184d admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:26.471542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 877/1747] 10.222.0.52 () {58 vars in 1018 bytes} [Sat Sep 24 14:26:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:26:26.523048 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c54835-9509-4adc-ae3d-a518f201c387 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:26.525130 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-94c54835-9509-4adc-ae3d-a518f201c387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:26.563361 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c54835-9509-4adc-ae3d-a518f201c387 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:26.583183 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8331cb1f-95d6-4558-9e70-be72cb2a0a2c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18fe384ecd454fb298dcc788f41dd44b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:26.583721 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-fecba004-d69f-40b3-ae4c-6b3096e9184d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c14990386754fd79905a63dd2606eb0'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:26.584733 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8331cb1f-95d6-4558-9e70-be72cb2a0a2c service placement] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:26.585677 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8331cb1f-95d6-4558-9e70-be72cb2a0a2c service placement] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:26.587555 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8331cb1f-95d6-4558-9e70-be72cb2a0a2c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:26.587884 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-fecba004-d69f-40b3-ae4c-6b3096e9184d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c14990386754fd79905a63dd2606eb0'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:26.602360 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-8331cb1f-95d6-4558-9e70-be72cb2a0a2c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:26.619701 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-fecba004-d69f-40b3-ae4c-6b3096e9184d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c14990386754fd79905a63dd2606eb0'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:26.620483 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-fecba004-d69f-40b3-ae4c-6b3096e9184d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c14990386754fd79905a63dd2606eb0'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:26.652503 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-fecba004-d69f-40b3-ae4c-6b3096e9184d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c14990386754fd79905a63dd2606eb0'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:26.665161 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8331cb1f-95d6-4558-9e70-be72cb2a0a2c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:26.670576 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8331cb1f-95d6-4558-9e70-be72cb2a0a2c service placement] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:26.681142 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:26.696832 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 871/1748] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:26:26 2022] DELETE /identity/v3/users/6c14990386754fd79905a63dd2606eb0 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:26:26.699159 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 878/1749] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 14:26:27.602903 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecba004-d69f-40b3-ae4c-6b3096e9184d admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:27.605211 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-fecba004-d69f-40b3-ae4c-6b3096e9184d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:27.692211 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8331cb1f-95d6-4558-9e70-be72cb2a0a2c service placement] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:27.693950 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-8331cb1f-95d6-4558-9e70-be72cb2a0a2c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:27.722015 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8331cb1f-95d6-4558-9e70-be72cb2a0a2c service placement] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:27.801229 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecba004-d69f-40b3-ae4c-6b3096e9184d admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:27.819762 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb4c5d-d2c9-4922-a721-d6d6147fa9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:27.820778 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb4c5d-d2c9-4922-a721-d6d6147fa9d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:27.821064 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb4c5d-d2c9-4922-a721-d6d6147fa9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:27.821335 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb4c5d-d2c9-4922-a721-d6d6147fa9d1 admin admin] PATH_INFO: `/v3/projects/39ed972dbf42400988bb9cdebfad21f5` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:27.822641 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb4c5d-d2c9-4922-a721-d6d6147fa9d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39ed972dbf42400988bb9cdebfad21f5)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:27.824613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb4c5d-d2c9-4922-a721-d6d6147fa9d1 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:27.873559 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76bbe0a-4d83-495b-89a2-5737efb766f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:27.876850 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76bbe0a-4d83-495b-89a2-5737efb766f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:27.877341 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76bbe0a-4d83-495b-89a2-5737efb766f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:27.877772 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76bbe0a-4d83-495b-89a2-5737efb766f6 admin admin] PATH_INFO: `/v3/projects/97bf63ef39ef400ab646dd447451aae2` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:27.881428 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76bbe0a-4d83-495b-89a2-5737efb766f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97bf63ef39ef400ab646dd447451aae2)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:27.885344 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76bbe0a-4d83-495b-89a2-5737efb766f6 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:27.924560 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-4ebb4c5d-d2c9-4922-a721-d6d6147fa9d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39ed972dbf42400988bb9cdebfad21f5', 'request_id': 'req-4ebb4c5d-d2c9-4922-a721-d6d6147fa9d1'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:27.940994 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-4ebb4c5d-d2c9-4922-a721-d6d6147fa9d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39ed972dbf42400988bb9cdebfad21f5', 'request_id': 'req-4ebb4c5d-d2c9-4922-a721-d6d6147fa9d1'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:27.951588 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-4ebb4c5d-d2c9-4922-a721-d6d6147fa9d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39ed972dbf42400988bb9cdebfad21f5', 'request_id': 'req-4ebb4c5d-d2c9-4922-a721-d6d6147fa9d1'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:27.976844 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-4ebb4c5d-d2c9-4922-a721-d6d6147fa9d1 admin admin] The token cache is being invalidate because project 39ed972dbf42400988bb9cdebfad21f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:26:27.977144 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-4ebb4c5d-d2c9-4922-a721-d6d6147fa9d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:27.980705 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-4ebb4c5d-d2c9-4922-a721-d6d6147fa9d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:27.990174 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 872/1750] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:26:27 2022] DELETE /identity/v3/projects/39ed972dbf42400988bb9cdebfad21f5 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:27.992072 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb4c5d-d2c9-4922-a721-d6d6147fa9d1 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:27.993332 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-4ebb4c5d-d2c9-4922-a721-d6d6147fa9d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:28.010761 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-f76bbe0a-4d83-495b-89a2-5737efb766f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97bf63ef39ef400ab646dd447451aae2', 'request_id': 'req-f76bbe0a-4d83-495b-89a2-5737efb766f6'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:28.019549 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-f76bbe0a-4d83-495b-89a2-5737efb766f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97bf63ef39ef400ab646dd447451aae2', 'request_id': 'req-f76bbe0a-4d83-495b-89a2-5737efb766f6'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:28.027371 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-f76bbe0a-4d83-495b-89a2-5737efb766f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97bf63ef39ef400ab646dd447451aae2', 'request_id': 'req-f76bbe0a-4d83-495b-89a2-5737efb766f6'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:28.073686 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-f76bbe0a-4d83-495b-89a2-5737efb766f6 admin admin] The token cache is being invalidate because project 97bf63ef39ef400ab646dd447451aae2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:26:28.074157 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-f76bbe0a-4d83-495b-89a2-5737efb766f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:28.075052 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-f76bbe0a-4d83-495b-89a2-5737efb766f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:28.080472 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 879/1751] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:26:27 2022] DELETE /identity/v3/projects/97bf63ef39ef400ab646dd447451aae2 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:28.081569 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76bbe0a-4d83-495b-89a2-5737efb766f6 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:28.084208 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-f76bbe0a-4d83-495b-89a2-5737efb766f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:28.175612 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:28.190516 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb4c5d-d2c9-4922-a721-d6d6147fa9d1 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:28.227876 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76bbe0a-4d83-495b-89a2-5737efb766f6 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:28.238187 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0427011e-1392-4805-b764-7cae534b28d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:28.240660 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427011e-1392-4805-b764-7cae534b28d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:28.242027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427011e-1392-4805-b764-7cae534b28d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:28.243818 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427011e-1392-4805-b764-7cae534b28d2 admin admin] PATH_INFO: `/v3/users/a9bd5d645cd444e7ba404060de7f0d7f` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:28.247426 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0427011e-1392-4805-b764-7cae534b28d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9bd5d645cd444e7ba404060de7f0d7f)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:28.250107 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0427011e-1392-4805-b764-7cae534b28d2 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:28.322973 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:28.392011 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-0427011e-1392-4805-b764-7cae534b28d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9bd5d645cd444e7ba404060de7f0d7f'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:28.393700 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-0427011e-1392-4805-b764-7cae534b28d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9bd5d645cd444e7ba404060de7f0d7f'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:28.394626 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-0427011e-1392-4805-b764-7cae534b28d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9bd5d645cd444e7ba404060de7f0d7f'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:28.407333 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e14652-5043-4641-b640-9e57a34cd249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:28.408510 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e14652-5043-4641-b640-9e57a34cd249 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:28.408510 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e14652-5043-4641-b640-9e57a34cd249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:28.408768 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e14652-5043-4641-b640-9e57a34cd249 admin admin] PATH_INFO: `/v3/users/c8ee73f10f174d1c95824575df1eb3a3` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:28.410341 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e14652-5043-4641-b640-9e57a34cd249 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8ee73f10f174d1c95824575df1eb3a3)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:28.412944 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e14652-5043-4641-b640-9e57a34cd249 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:28.426680 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-0427011e-1392-4805-b764-7cae534b28d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9bd5d645cd444e7ba404060de7f0d7f'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:28.449007 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-0427011e-1392-4805-b764-7cae534b28d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9bd5d645cd444e7ba404060de7f0d7f'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:28.471209 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 880/1752] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:26:28 2022] DELETE /identity/v3/users/a9bd5d645cd444e7ba404060de7f0d7f => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:28.491225 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ae1d4-de90-4647-8688-40dee0b731ed None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:28.491225 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ae1d4-de90-4647-8688-40dee0b731ed None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:28.491225 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ae1d4-de90-4647-8688-40dee0b731ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:28.563848 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-9d0ae1d4-de90-4647-8688-40dee0b731ed None None] MFA Rules not processed for user `883e6570afb84a099e46cdace5a88b2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:28.572556 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-9d0ae1d4-de90-4647-8688-40dee0b731ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:28.573856 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-67e14652-5043-4641-b640-9e57a34cd249 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8ee73f10f174d1c95824575df1eb3a3'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:28.576924 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-67e14652-5043-4641-b640-9e57a34cd249 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8ee73f10f174d1c95824575df1eb3a3'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:28.642982 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-67e14652-5043-4641-b640-9e57a34cd249 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8ee73f10f174d1c95824575df1eb3a3'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:28.650866 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-67e14652-5043-4641-b640-9e57a34cd249 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8ee73f10f174d1c95824575df1eb3a3'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:28.720263 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-67e14652-5043-4641-b640-9e57a34cd249 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8ee73f10f174d1c95824575df1eb3a3'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:28.765946 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 873/1753] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:26:27 2022] DELETE /identity/v3/users/c8ee73f10f174d1c95824575df1eb3a3 => generated 0 bytes in 775 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:26:28.822458 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 881/1754] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:28 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:26:29.025051 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e14652-5043-4641-b640-9e57a34cd249 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:29.027782 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-67e14652-5043-4641-b640-9e57a34cd249 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:29.249206 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e14652-5043-4641-b640-9e57a34cd249 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:29.280744 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0137f8-f97a-4362-bcf7-c4203341af81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:29.284518 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0137f8-f97a-4362-bcf7-c4203341af81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:29.286466 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0137f8-f97a-4362-bcf7-c4203341af81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:29.287557 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0137f8-f97a-4362-bcf7-c4203341af81 admin admin] PATH_INFO: `/v3/projects/5f17177494944cfb9602cd48a91a2d34` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:29.290787 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0137f8-f97a-4362-bcf7-c4203341af81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f17177494944cfb9602cd48a91a2d34)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:29.298666 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0137f8-f97a-4362-bcf7-c4203341af81 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:29.506727 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-2a0137f8-f97a-4362-bcf7-c4203341af81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f17177494944cfb9602cd48a91a2d34', 'request_id': 'req-2a0137f8-f97a-4362-bcf7-c4203341af81'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:29.518885 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-2a0137f8-f97a-4362-bcf7-c4203341af81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f17177494944cfb9602cd48a91a2d34', 'request_id': 'req-2a0137f8-f97a-4362-bcf7-c4203341af81'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:29.524729 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:29.531340 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-2a0137f8-f97a-4362-bcf7-c4203341af81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f17177494944cfb9602cd48a91a2d34', 'request_id': 'req-2a0137f8-f97a-4362-bcf7-c4203341af81'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:29.576514 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-2a0137f8-f97a-4362-bcf7-c4203341af81 admin admin] The token cache is being invalidate because project 5f17177494944cfb9602cd48a91a2d34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:26:29.578335 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-2a0137f8-f97a-4362-bcf7-c4203341af81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:29.580884 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-2a0137f8-f97a-4362-bcf7-c4203341af81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:29.585883 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 874/1755] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:26:29 2022] DELETE /identity/v3/projects/5f17177494944cfb9602cd48a91a2d34 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:29.591292 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c4e70-c936-4780-8fb5-a91f3f34c738 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:29.591576 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c4e70-c936-4780-8fb5-a91f3f34c738 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:29.591854 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c4e70-c936-4780-8fb5-a91f3f34c738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:29.644870 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-323c4e70-c936-4780-8fb5-a91f3f34c738 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:29.646108 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-323c4e70-c936-4780-8fb5-a91f3f34c738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:29.810421 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 882/1756] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:29.818852 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42c7f4-1760-4150-8fbd-ae17fdfd84b7 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:29.819944 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42c7f4-1760-4150-8fbd-ae17fdfd84b7 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:29.820386 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42c7f4-1760-4150-8fbd-ae17fdfd84b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:29.875778 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-5d42c7f4-1760-4150-8fbd-ae17fdfd84b7 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:29.879171 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-5d42c7f4-1760-4150-8fbd-ae17fdfd84b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:29.895349 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 875/1757] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:29.900801 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c4e70-c936-4780-8fb5-a91f3f34c738 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:29.917144 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:29.940948 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c4e70-c936-4780-8fb5-a91f3f34c738 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:29.956691 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc6a40-76fb-466b-bd96-830100b4ed40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:29.969922 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc6a40-76fb-466b-bd96-830100b4ed40 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:29.972759 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc6a40-76fb-466b-bd96-830100b4ed40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:29.973141 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc6a40-76fb-466b-bd96-830100b4ed40 admin admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:29.976580 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc6a40-76fb-466b-bd96-830100b4ed40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:26:29.977438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc6a40-76fb-466b-bd96-830100b4ed40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:29.981297 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc6a40-76fb-466b-bd96-830100b4ed40 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:30.041062 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 883/1758] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:26:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:26:30.041502 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d42c7f4-1760-4150-8fbd-ae17fdfd84b7 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:30.063697 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d42c7f4-1760-4150-8fbd-ae17fdfd84b7 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:30.091167 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1baadc-bce8-4171-a613-5f74aaeaf1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:30.097013 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1baadc-bce8-4171-a613-5f74aaeaf1ad admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:30.102960 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1baadc-bce8-4171-a613-5f74aaeaf1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:30.108902 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1baadc-bce8-4171-a613-5f74aaeaf1ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:30.108902 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1baadc-bce8-4171-a613-5f74aaeaf1ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:30.108902 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1baadc-bce8-4171-a613-5f74aaeaf1ad admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:30.135852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c45b6-2ba9-4ca4-8fa1-01fc81cbae3f None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:30.136057 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c45b6-2ba9-4ca4-8fa1-01fc81cbae3f None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:30.136296 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c45b6-2ba9-4ca4-8fa1-01fc81cbae3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:30.262871 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-f42c45b6-2ba9-4ca4-8fa1-01fc81cbae3f None None] MFA Rules not processed for user `a9348164132846d2b593ddfa29a820eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:30.275351 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-f42c45b6-2ba9-4ca4-8fa1-01fc81cbae3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:30.322423 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 876/1759] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:26:30 2022] POST /identity/v3/projects => generated 365 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:30.325505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1baadc-bce8-4171-a613-5f74aaeaf1ad admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:30.603497 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 884/1760] 10.222.0.33 () {58 vars in 1015 bytes} [Sat Sep 24 14:26:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 470 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:30.675700 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1baadc-bce8-4171-a613-5f74aaeaf1ad admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:30.699067 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c45b6-2ba9-4ca4-8fa1-01fc81cbae3f None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:30.699067 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-f42c45b6-2ba9-4ca4-8fa1-01fc81cbae3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:30.705629 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd16a6-f79d-413f-addc-b17da995add9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:30.705629 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd16a6-f79d-413f-addc-b17da995add9 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:30.705629 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd16a6-f79d-413f-addc-b17da995add9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:30.705629 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd16a6-f79d-413f-addc-b17da995add9 admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:30.705926 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fd16a6-f79d-413f-addc-b17da995add9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:30.705926 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fd16a6-f79d-413f-addc-b17da995add9 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:30.722471 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c45b6-2ba9-4ca4-8fa1-01fc81cbae3f None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:30.813636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 877/1761] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:26:30 2022] POST /identity/v3/users => generated 283 bytes in 491 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:30.820058 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd16a6-f79d-413f-addc-b17da995add9 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:30.862339 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd16a6-f79d-413f-addc-b17da995add9 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:30.876121 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1723592a-51b1-43ee-b233-3a146d4ae033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:30.876756 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723592a-51b1-43ee-b233-3a146d4ae033 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:30.877032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723592a-51b1-43ee-b233-3a146d4ae033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:30.877302 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723592a-51b1-43ee-b233-3a146d4ae033 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:30.892315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1723592a-51b1-43ee-b233-3a146d4ae033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:30.896457 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1723592a-51b1-43ee-b233-3a146d4ae033 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:30.934510 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 878/1762] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:30.935598 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1723592a-51b1-43ee-b233-3a146d4ae033 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:30.954770 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198049b-8eb1-4556-9b81-d9cbb6a4d668 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:30.955467 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198049b-8eb1-4556-9b81-d9cbb6a4d668 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:30.955747 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198049b-8eb1-4556-9b81-d9cbb6a4d668 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:30.956020 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198049b-8eb1-4556-9b81-d9cbb6a4d668 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:30.972355 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1723592a-51b1-43ee-b233-3a146d4ae033 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:30.976660 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0198049b-8eb1-4556-9b81-d9cbb6a4d668 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:30.978128 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0198049b-8eb1-4556-9b81-d9cbb6a4d668 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:30.985666 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72128b7-7573-4027-b4f0-edcc596192f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:30.986294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72128b7-7573-4027-b4f0-edcc596192f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:30.986571 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72128b7-7573-4027-b4f0-edcc596192f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:30.986879 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72128b7-7573-4027-b4f0-edcc596192f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:30.987531 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72128b7-7573-4027-b4f0-edcc596192f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:30.989139 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72128b7-7573-4027-b4f0-edcc596192f3 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:31.005229 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 885/1763] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:31.046395 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 879/1764] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:31.048098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198049b-8eb1-4556-9b81-d9cbb6a4d668 service neutron] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:31.075961 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198049b-8eb1-4556-9b81-d9cbb6a4d668 service neutron] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:31.101387 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2935b2fe-27e1-4fc2-a40e-8cce9e8fd7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:31.103860 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935b2fe-27e1-4fc2-a40e-8cce9e8fd7df admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:31.104146 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935b2fe-27e1-4fc2-a40e-8cce9e8fd7df admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:31.106876 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935b2fe-27e1-4fc2-a40e-8cce9e8fd7df admin admin] PATH_INFO: `/v3/projects/c482609d93e840bf8d1be3a03513b91d/users/b78bb1ccf4dc4ea2841073e502711ca4/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:31.153225 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2935b2fe-27e1-4fc2-a40e-8cce9e8fd7df admin admin] RBAC: Authorizing `identity:create_grant(project_id=c482609d93e840bf8d1be3a03513b91d, user_id=b78bb1ccf4dc4ea2841073e502711ca4, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:31.163068 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2935b2fe-27e1-4fc2-a40e-8cce9e8fd7df admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:31.193583 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 886/1765] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:31 2022] PUT /identity/v3/projects/c482609d93e840bf8d1be3a03513b91d/users/b78bb1ccf4dc4ea2841073e502711ca4/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:26:31.209117 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf276ae0-ec22-4e61-a6da-90ff585b5dc0 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:31.209457 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf276ae0-ec22-4e61-a6da-90ff585b5dc0 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:31.209731 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf276ae0-ec22-4e61-a6da-90ff585b5dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:31.315955 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-cf276ae0-ec22-4e61-a6da-90ff585b5dc0 None None] MFA Rules not processed for user `b78bb1ccf4dc4ea2841073e502711ca4`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:31.317521 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-cf276ae0-ec22-4e61-a6da-90ff585b5dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:31.441317 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2935b2fe-27e1-4fc2-a40e-8cce9e8fd7df admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:31.443818 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-2935b2fe-27e1-4fc2-a40e-8cce9e8fd7df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:31.599248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 880/1766] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:31.602671 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf276ae0-ec22-4e61-a6da-90ff585b5dc0 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:31.609770 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2935b2fe-27e1-4fc2-a40e-8cce9e8fd7df admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:31.636450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f5cc4-5089-49a2-9c98-531d5ebd2997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:31.636871 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf276ae0-ec22-4e61-a6da-90ff585b5dc0 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:31.637110 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f5cc4-5089-49a2-9c98-531d5ebd2997 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:31.637391 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f5cc4-5089-49a2-9c98-531d5ebd2997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:31.637664 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f5cc4-5089-49a2-9c98-531d5ebd2997 admin admin] PATH_INFO: `/v3/projects/ef05a1a524df41b4adbd8f0fcde5725a` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:31.642321 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259f5cc4-5089-49a2-9c98-531d5ebd2997 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef05a1a524df41b4adbd8f0fcde5725a)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:31.643296 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64979f1-3bd8-48c7-a216-c3d152cbad3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:31.644307 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259f5cc4-5089-49a2-9c98-531d5ebd2997 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:31.644577 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64979f1-3bd8-48c7-a216-c3d152cbad3b admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:31.645340 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64979f1-3bd8-48c7-a216-c3d152cbad3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:31.645340 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64979f1-3bd8-48c7-a216-c3d152cbad3b admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:31.646063 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64979f1-3bd8-48c7-a216-c3d152cbad3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:31.650875 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64979f1-3bd8-48c7-a216-c3d152cbad3b admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:31.732424 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:31.752111 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:31.802695 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 881/1767] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:26:31 2022] POST /identity/v3/projects => generated 365 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:31.808634 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64979f1-3bd8-48c7-a216-c3d152cbad3b admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:31.885775 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-259f5cc4-5089-49a2-9c98-531d5ebd2997 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef05a1a524df41b4adbd8f0fcde5725a', 'request_id': 'req-259f5cc4-5089-49a2-9c98-531d5ebd2997'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:31.916395 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-259f5cc4-5089-49a2-9c98-531d5ebd2997 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef05a1a524df41b4adbd8f0fcde5725a', 'request_id': 'req-259f5cc4-5089-49a2-9c98-531d5ebd2997'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:31.951278 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-259f5cc4-5089-49a2-9c98-531d5ebd2997 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef05a1a524df41b4adbd8f0fcde5725a', 'request_id': 'req-259f5cc4-5089-49a2-9c98-531d5ebd2997'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:32.030739 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64979f1-3bd8-48c7-a216-c3d152cbad3b admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:32.047484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-259f5cc4-5089-49a2-9c98-531d5ebd2997 admin admin] The token cache is being invalidate because project ef05a1a524df41b4adbd8f0fcde5725a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:26:32.050985 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762644bc-87fb-41ad-8f1c-0eda695289d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:32.051299 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-259f5cc4-5089-49a2-9c98-531d5ebd2997 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:32.052842 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762644bc-87fb-41ad-8f1c-0eda695289d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:32.053127 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762644bc-87fb-41ad-8f1c-0eda695289d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:32.055336 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-259f5cc4-5089-49a2-9c98-531d5ebd2997 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:32.055506 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762644bc-87fb-41ad-8f1c-0eda695289d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:32.056890 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762644bc-87fb-41ad-8f1c-0eda695289d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:32.062912 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 887/1768] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:26:31 2022] DELETE /identity/v3/projects/ef05a1a524df41b4adbd8f0fcde5725a => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:32.066751 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762644bc-87fb-41ad-8f1c-0eda695289d9 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:32.067124 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24acf3f3-e4d7-470f-ae33-5afc609e545b None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:32.069690 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24acf3f3-e4d7-470f-ae33-5afc609e545b None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:32.070137 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24acf3f3-e4d7-470f-ae33-5afc609e545b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:32.143754 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-24acf3f3-e4d7-470f-ae33-5afc609e545b None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:32.145768 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-24acf3f3-e4d7-470f-ae33-5afc609e545b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:32.159688 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 882/1769] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:26:31 2022] POST /identity/v3/users => generated 289 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:32.172480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 888/1770] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:32.179922 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca83575-064c-4993-ab41-a25763581c79 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:32.180173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762644bc-87fb-41ad-8f1c-0eda695289d9 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:32.180595 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca83575-064c-4993-ab41-a25763581c79 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:32.181016 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca83575-064c-4993-ab41-a25763581c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:32.187133 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-762644bc-87fb-41ad-8f1c-0eda695289d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:32.231437 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762644bc-87fb-41ad-8f1c-0eda695289d9 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:32.245837 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c335a-42c3-4b94-8072-e8a318fdbb5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:32.246717 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c335a-42c3-4b94-8072-e8a318fdbb5f admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:32.248971 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c335a-42c3-4b94-8072-e8a318fdbb5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:32.250404 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c335a-42c3-4b94-8072-e8a318fdbb5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:32.252482 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318c335a-42c3-4b94-8072-e8a318fdbb5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:32.263155 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318c335a-42c3-4b94-8072-e8a318fdbb5f admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:32.264349 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-aca83575-064c-4993-ab41-a25763581c79 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:32.267642 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-aca83575-064c-4993-ab41-a25763581c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:32.298269 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 889/1771] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:32.299288 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca83575-064c-4993-ab41-a25763581c79 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:32.306495 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 883/1772] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:32.307656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c335a-42c3-4b94-8072-e8a318fdbb5f admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:32.322985 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c335a-42c3-4b94-8072-e8a318fdbb5f admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:32.336396 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca83575-064c-4993-ab41-a25763581c79 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:32.352363 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e96e2-97ee-44ab-935f-2bc5a6761bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:32.353145 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e96e2-97ee-44ab-935f-2bc5a6761bcf admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:32.354779 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e96e2-97ee-44ab-935f-2bc5a6761bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:32.357316 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e96e2-97ee-44ab-935f-2bc5a6761bcf admin admin] PATH_INFO: `/v3/projects/70db651bf80c4b43b5f06f356dc2e539/users/8394034b874f409e86dd56986e98da3a/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:32.359600 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc01023-5c63-445d-b297-beb047bb909d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:32.360408 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc01023-5c63-445d-b297-beb047bb909d admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:32.360845 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc01023-5c63-445d-b297-beb047bb909d admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:32.362117 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc01023-5c63-445d-b297-beb047bb909d admin admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:32.365676 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc01023-5c63-445d-b297-beb047bb909d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:26:32.366017 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc01023-5c63-445d-b297-beb047bb909d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:32.375470 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc01023-5c63-445d-b297-beb047bb909d admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:32.384572 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264e96e2-97ee-44ab-935f-2bc5a6761bcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=70db651bf80c4b43b5f06f356dc2e539, user_id=8394034b874f409e86dd56986e98da3a, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:32.388961 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264e96e2-97ee-44ab-935f-2bc5a6761bcf admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:32.432716 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 890/1773] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:26:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:26:32.451912 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 884/1774] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:32 2022] PUT /identity/v3/projects/70db651bf80c4b43b5f06f356dc2e539/users/8394034b874f409e86dd56986e98da3a/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:32.452070 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc01023-5c63-445d-b297-beb047bb909d admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:32.453228 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e96e2-97ee-44ab-935f-2bc5a6761bcf admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:32.500243 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc01023-5c63-445d-b297-beb047bb909d admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:32.697687 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e96e2-97ee-44ab-935f-2bc5a6761bcf admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:32.711440 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eecb8d4-bc95-40f6-b594-bf8e6b38ce6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:32.712497 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecb8d4-bc95-40f6-b594-bf8e6b38ce6a admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:32.713586 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecb8d4-bc95-40f6-b594-bf8e6b38ce6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:32.714567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad004b-3f56-496e-9913-cb5e3a1aa590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:32.715089 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecb8d4-bc95-40f6-b594-bf8e6b38ce6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:32.716320 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eecb8d4-bc95-40f6-b594-bf8e6b38ce6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:32.716637 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad004b-3f56-496e-9913-cb5e3a1aa590 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:32.717053 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad004b-3f56-496e-9913-cb5e3a1aa590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:32.717742 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad004b-3f56-496e-9913-cb5e3a1aa590 admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:32.719304 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eecb8d4-bc95-40f6-b594-bf8e6b38ce6a admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:32.720341 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ad004b-3f56-496e-9913-cb5e3a1aa590 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:32.722576 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ad004b-3f56-496e-9913-cb5e3a1aa590 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:32.758043 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 885/1775] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:32.758043 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eecb8d4-bc95-40f6-b594-bf8e6b38ce6a admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:32.782338 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eecb8d4-bc95-40f6-b594-bf8e6b38ce6a admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:32.792569 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f4e16c-b23a-4413-bff9-dba508b21e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:32.793797 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f4e16c-b23a-4413-bff9-dba508b21e2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:32.794230 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f4e16c-b23a-4413-bff9-dba508b21e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:32.794681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f4e16c-b23a-4413-bff9-dba508b21e2b admin admin] PATH_INFO: `/v3/projects/70db651bf80c4b43b5f06f356dc2e539/users/8394034b874f409e86dd56986e98da3a/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:32.800859 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f4e16c-b23a-4413-bff9-dba508b21e2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=70db651bf80c4b43b5f06f356dc2e539, user_id=8394034b874f409e86dd56986e98da3a, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:32.810160 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f4e16c-b23a-4413-bff9-dba508b21e2b admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:32.812193 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 891/1776] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:26:32 2022] POST /identity/v3/projects => generated 375 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:32.819520 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad004b-3f56-496e-9913-cb5e3a1aa590 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:32.855923 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 886/1777] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:32 2022] PUT /identity/v3/projects/70db651bf80c4b43b5f06f356dc2e539/users/8394034b874f409e86dd56986e98da3a/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:32.861053 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38e61a-fc4c-4870-aae3-a7221d578fff None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:32.861616 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38e61a-fc4c-4870-aae3-a7221d578fff None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:32.863321 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38e61a-fc4c-4870-aae3-a7221d578fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:33.007919 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-ea38e61a-fc4c-4870-aae3-a7221d578fff None None] MFA Rules not processed for user `8394034b874f409e86dd56986e98da3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:33.013306 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-ea38e61a-fc4c-4870-aae3-a7221d578fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:33.095550 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad004b-3f56-496e-9913-cb5e3a1aa590 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:33.139693 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:33.149850 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:33.321282 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 887/1778] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:32 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 464 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:33.352107 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38e61a-fc4c-4870-aae3-a7221d578fff None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:33.354645 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-ea38e61a-fc4c-4870-aae3-a7221d578fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:33.377471 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20788617-6426-4659-ad64-d0b85498898c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:33.386722 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20788617-6426-4659-ad64-d0b85498898c admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:33.392101 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20788617-6426-4659-ad64-d0b85498898c admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:33.392410 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20788617-6426-4659-ad64-d0b85498898c admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:33.393094 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20788617-6426-4659-ad64-d0b85498898c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:33.395334 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20788617-6426-4659-ad64-d0b85498898c admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:33.463039 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 892/1779] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:26:32 2022] POST /identity/v3/users => generated 288 bytes in 641 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:33.465178 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20788617-6426-4659-ad64-d0b85498898c admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:33.494554 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20788617-6426-4659-ad64-d0b85498898c admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:33.510799 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496579dd-501f-4819-bbc2-f7e103454d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:33.511566 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496579dd-501f-4819-bbc2-f7e103454d44 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:33.511924 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496579dd-501f-4819-bbc2-f7e103454d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:33.512275 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496579dd-501f-4819-bbc2-f7e103454d44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:33.513017 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496579dd-501f-4819-bbc2-f7e103454d44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:33.514717 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496579dd-501f-4819-bbc2-f7e103454d44 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:33.547135 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38e61a-fc4c-4870-aae3-a7221d578fff None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:33.552701 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 893/1780] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:33.557786 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496579dd-501f-4819-bbc2-f7e103454d44 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:33.574495 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d8c8b-f804-4dc0-bced-e95b81b391a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:33.583266 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d8c8b-f804-4dc0-bced-e95b81b391a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:33.583663 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d8c8b-f804-4dc0-bced-e95b81b391a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:33.583861 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d8c8b-f804-4dc0-bced-e95b81b391a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:33.585481 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496579dd-501f-4819-bbc2-f7e103454d44 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:33.600469 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60b3989-1f31-457a-9e5f-645c351dd4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:33.601958 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b3989-1f31-457a-9e5f-645c351dd4fd admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:33.601958 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b3989-1f31-457a-9e5f-645c351dd4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:33.602302 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b3989-1f31-457a-9e5f-645c351dd4fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:33.603027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60b3989-1f31-457a-9e5f-645c351dd4fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:33.613783 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60b3989-1f31-457a-9e5f-645c351dd4fd admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:33.637344 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 894/1781] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:33.642094 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60b3989-1f31-457a-9e5f-645c351dd4fd admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:33.647738 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223d8c8b-f804-4dc0-bced-e95b81b391a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:33.659375 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223d8c8b-f804-4dc0-bced-e95b81b391a9 service cinder] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:33.662504 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60b3989-1f31-457a-9e5f-645c351dd4fd admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:33.672914 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6249bf-88ea-413d-956d-62718ec75458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:33.674060 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6249bf-88ea-413d-956d-62718ec75458 admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:33.674456 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6249bf-88ea-413d-956d-62718ec75458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:33.674797 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6249bf-88ea-413d-956d-62718ec75458 admin admin] PATH_INFO: `/v3/projects/eda57b62d6bc44928579c7a35f3e251d/users/8af199617ed5413b96728d1171892bbf/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:33.715207 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 888/1782] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:33 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 14:26:33.719624 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6249bf-88ea-413d-956d-62718ec75458 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eda57b62d6bc44928579c7a35f3e251d, user_id=8af199617ed5413b96728d1171892bbf, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:33.723339 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6249bf-88ea-413d-956d-62718ec75458 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:33.762844 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 895/1783] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:33 2022] PUT /identity/v3/projects/eda57b62d6bc44928579c7a35f3e251d/users/8af199617ed5413b96728d1171892bbf/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:33.775082 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b8033-15d0-4185-9aa6-953c1359c249 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:33.775394 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b8033-15d0-4185-9aa6-953c1359c249 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:33.775813 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b8033-15d0-4185-9aa6-953c1359c249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:33.898232 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-4a6b8033-15d0-4185-9aa6-953c1359c249 None None] MFA Rules not processed for user `8af199617ed5413b96728d1171892bbf`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:33.899443 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-4a6b8033-15d0-4185-9aa6-953c1359c249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:33.994896 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b99d72-4367-4226-8852-5061d64d27a6 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:33.995983 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b99d72-4367-4226-8852-5061d64d27a6 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:33.998132 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b99d72-4367-4226-8852-5061d64d27a6 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:34.001632 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 896/1784] 10.222.0.52 () {54 vars in 921 bytes} [Sat Sep 24 14:26:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:26:34.060873 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa00d8-848c-4ec6-b446-d1836b0c10c9 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:34.061414 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa00d8-848c-4ec6-b446-d1836b0c10c9 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:34.061862 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa00d8-848c-4ec6-b446-d1836b0c10c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:34.128577 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-9daa00d8-848c-4ec6-b446-d1836b0c10c9 None None] MFA Rules not processed for user `79eaaae97aac4282a613939e5b768376`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:34.134422 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 889/1785] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:34.134505 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-9daa00d8-848c-4ec6-b446-d1836b0c10c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:34.134595 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6b8033-15d0-4185-9aa6-953c1359c249 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:34.308334 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:34.369955 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6b8033-15d0-4185-9aa6-953c1359c249 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:34.377395 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8397df-500b-40b3-8493-7dd42ba03be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:34.378129 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8397df-500b-40b3-8493-7dd42ba03be2 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:34.379700 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8397df-500b-40b3-8493-7dd42ba03be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:34.380029 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8397df-500b-40b3-8493-7dd42ba03be2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:34.380785 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8397df-500b-40b3-8493-7dd42ba03be2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:34.383920 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8397df-500b-40b3-8493-7dd42ba03be2 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:34.391630 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 897/1786] 10.222.0.52 () {58 vars in 1018 bytes} [Sat Sep 24 14:26:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:34.496886 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa00d8-848c-4ec6-b446-d1836b0c10c9 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:34.499912 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-9daa00d8-848c-4ec6-b446-d1836b0c10c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:34.504903 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 890/1787] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:26:34 2022] POST /identity/v3/projects => generated 377 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:26:34.507075 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8397df-500b-40b3-8493-7dd42ba03be2 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:34.542642 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa00d8-848c-4ec6-b446-d1836b0c10c9 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:34.703829 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8397df-500b-40b3-8493-7dd42ba03be2 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:34.706986 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:34.723938 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5adfa12-5b62-4657-a6de-10a1feb1ef1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:34.724588 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5adfa12-5b62-4657-a6de-10a1feb1ef1f admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:34.724909 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5adfa12-5b62-4657-a6de-10a1feb1ef1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:34.725159 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5adfa12-5b62-4657-a6de-10a1feb1ef1f admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:34.726339 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5adfa12-5b62-4657-a6de-10a1feb1ef1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:34.731391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5adfa12-5b62-4657-a6de-10a1feb1ef1f admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:34.744387 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b91b1-207e-4c1f-83c4-58e30a21174c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:34.745246 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b91b1-207e-4c1f-83c4-58e30a21174c service nova] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:34.745685 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b91b1-207e-4c1f-83c4-58e30a21174c service nova] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:34.746118 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b91b1-207e-4c1f-83c4-58e30a21174c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:34.767740 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b91b1-207e-4c1f-83c4-58e30a21174c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:34.769594 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b91b1-207e-4c1f-83c4-58e30a21174c service nova] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:34.788416 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 898/1788] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:34.849823 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 891/1789] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:26:34 2022] POST /identity/v3/users => generated 295 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:34.854425 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b91b1-207e-4c1f-83c4-58e30a21174c service nova] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:34.870309 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b91b1-207e-4c1f-83c4-58e30a21174c service nova] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:34.884031 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba2405c-fe39-4422-8f21-c3f93d16100a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:34.884675 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba2405c-fe39-4422-8f21-c3f93d16100a admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:34.884977 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba2405c-fe39-4422-8f21-c3f93d16100a admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:34.885227 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba2405c-fe39-4422-8f21-c3f93d16100a admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:34.885876 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba2405c-fe39-4422-8f21-c3f93d16100a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:34.888833 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba2405c-fe39-4422-8f21-c3f93d16100a admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:34.917045 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5adfa12-5b62-4657-a6de-10a1feb1ef1f admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:34.928279 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 899/1790] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:34 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:34.940922 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5adfa12-5b62-4657-a6de-10a1feb1ef1f admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:34.956270 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a441cc-8f49-4168-83e9-d333eeedceb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:34.956963 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a441cc-8f49-4168-83e9-d333eeedceb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:34.957242 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a441cc-8f49-4168-83e9-d333eeedceb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:34.957523 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a441cc-8f49-4168-83e9-d333eeedceb1 admin admin] PATH_INFO: `/v3/projects/1ef76f16c463457e968dce2749a14fe8/users/2e70ba3e1f6a42e3bc3dec61232c6c02/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:35.019296 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a441cc-8f49-4168-83e9-d333eeedceb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ef76f16c463457e968dce2749a14fe8, user_id=2e70ba3e1f6a42e3bc3dec61232c6c02, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:35.028065 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a441cc-8f49-4168-83e9-d333eeedceb1 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:35.061406 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 892/1791] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:34 2022] PUT /identity/v3/projects/1ef76f16c463457e968dce2749a14fe8/users/2e70ba3e1f6a42e3bc3dec61232c6c02/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:26:35.065471 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba2405c-fe39-4422-8f21-c3f93d16100a admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:35.081984 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba2405c-fe39-4422-8f21-c3f93d16100a admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:35.311022 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9774b7-6b24-417c-bb0d-fc66352eb96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:35.311762 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9774b7-6b24-417c-bb0d-fc66352eb96b admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:35.312443 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9774b7-6b24-417c-bb0d-fc66352eb96b admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:35.312876 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9774b7-6b24-417c-bb0d-fc66352eb96b admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:35.313755 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9774b7-6b24-417c-bb0d-fc66352eb96b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:35.330328 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9774b7-6b24-417c-bb0d-fc66352eb96b admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:35.359571 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 900/1792] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:35.363790 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a441cc-8f49-4168-83e9-d333eeedceb1 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:35.421922 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a441cc-8f49-4168-83e9-d333eeedceb1 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:35.434552 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a44d2a4-3d6b-4e3c-b267-3c83b3f27e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:35.434552 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44d2a4-3d6b-4e3c-b267-3c83b3f27e0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:35.434552 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44d2a4-3d6b-4e3c-b267-3c83b3f27e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:35.434552 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44d2a4-3d6b-4e3c-b267-3c83b3f27e0b admin admin] PATH_INFO: `/v3/projects/1ef76f16c463457e968dce2749a14fe8/users/2e70ba3e1f6a42e3bc3dec61232c6c02/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:35.434923 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a44d2a4-3d6b-4e3c-b267-3c83b3f27e0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ef76f16c463457e968dce2749a14fe8, user_id=2e70ba3e1f6a42e3bc3dec61232c6c02, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:35.437860 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a44d2a4-3d6b-4e3c-b267-3c83b3f27e0b admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:35.473294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 893/1793] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:35 2022] PUT /identity/v3/projects/1ef76f16c463457e968dce2749a14fe8/users/2e70ba3e1f6a42e3bc3dec61232c6c02/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:35.476462 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41c252-8cab-48d8-9353-a2b6150e7246 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:35.476462 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41c252-8cab-48d8-9353-a2b6150e7246 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:35.477315 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41c252-8cab-48d8-9353-a2b6150e7246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:35.528047 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-2c41c252-8cab-48d8-9353-a2b6150e7246 None None] MFA Rules not processed for user `2e70ba3e1f6a42e3bc3dec61232c6c02`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:35.529550 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-2c41c252-8cab-48d8-9353-a2b6150e7246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:35.651658 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 901/1794] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:35 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:35.767754 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a44d2a4-3d6b-4e3c-b267-3c83b3f27e0b admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:35.870399 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a44d2a4-3d6b-4e3c-b267-3c83b3f27e0b admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:35.881819 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb28e79-0dd4-45b9-a9ea-dc100f78a69d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:35.883401 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb28e79-0dd4-45b9-a9ea-dc100f78a69d service cinder] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:35.883692 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb28e79-0dd4-45b9-a9ea-dc100f78a69d service cinder] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:35.883967 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb28e79-0dd4-45b9-a9ea-dc100f78a69d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:35.947870 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:35.974119 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb28e79-0dd4-45b9-a9ea-dc100f78a69d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:35.978127 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb28e79-0dd4-45b9-a9ea-dc100f78a69d service cinder] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:36.001975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 894/1795] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:36.111419 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41c252-8cab-48d8-9353-a2b6150e7246 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:36.240277 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:36.275785 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb28e79-0dd4-45b9-a9ea-dc100f78a69d service cinder] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:36.277598 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-eeb28e79-0dd4-45b9-a9ea-dc100f78a69d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:36.292824 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41c252-8cab-48d8-9353-a2b6150e7246 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:36.310102 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e5c55-b910-4f3b-b262-46cc624979c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:36.311470 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e5c55-b910-4f3b-b262-46cc624979c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:36.312093 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb28e79-0dd4-45b9-a9ea-dc100f78a69d service cinder] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:36.312184 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e5c55-b910-4f3b-b262-46cc624979c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:36.312184 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e5c55-b910-4f3b-b262-46cc624979c9 admin admin] PATH_INFO: `/v3/users/b78bb1ccf4dc4ea2841073e502711ca4` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:36.313205 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266e5c55-b910-4f3b-b262-46cc624979c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b78bb1ccf4dc4ea2841073e502711ca4)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:36.315248 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266e5c55-b910-4f3b-b262-46cc624979c9 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:36.318601 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b885c3cc-8db3-47b5-bf63-79752042999a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:36.320089 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885c3cc-8db3-47b5-bf63-79752042999a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:36.320374 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885c3cc-8db3-47b5-bf63-79752042999a admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:36.320647 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885c3cc-8db3-47b5-bf63-79752042999a admin admin] PATH_INFO: `/v3/users/a60ca6fcc27e49bdba767217a0e537c7` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:36.322067 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b885c3cc-8db3-47b5-bf63-79752042999a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a60ca6fcc27e49bdba767217a0e537c7)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:36.324581 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b885c3cc-8db3-47b5-bf63-79752042999a admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:36.414945 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-266e5c55-b910-4f3b-b262-46cc624979c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b78bb1ccf4dc4ea2841073e502711ca4'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:36.415621 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-266e5c55-b910-4f3b-b262-46cc624979c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b78bb1ccf4dc4ea2841073e502711ca4'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:36.416278 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-266e5c55-b910-4f3b-b262-46cc624979c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b78bb1ccf4dc4ea2841073e502711ca4'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:36.433239 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-b885c3cc-8db3-47b5-bf63-79752042999a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a60ca6fcc27e49bdba767217a0e537c7'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:36.434595 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-b885c3cc-8db3-47b5-bf63-79752042999a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a60ca6fcc27e49bdba767217a0e537c7'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:36.435605 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-266e5c55-b910-4f3b-b262-46cc624979c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b78bb1ccf4dc4ea2841073e502711ca4'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:36.454872 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-266e5c55-b910-4f3b-b262-46cc624979c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b78bb1ccf4dc4ea2841073e502711ca4'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:36.459222 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-b885c3cc-8db3-47b5-bf63-79752042999a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a60ca6fcc27e49bdba767217a0e537c7'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:36.460813 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-b885c3cc-8db3-47b5-bf63-79752042999a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a60ca6fcc27e49bdba767217a0e537c7'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:36.496741 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 902/1796] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:26:36 2022] DELETE /identity/v3/users/b78bb1ccf4dc4ea2841073e502711ca4 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:36.502479 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73f301-327c-4108-ac31-ab8bc7e0fd03 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:36.502775 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73f301-327c-4108-ac31-ab8bc7e0fd03 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:36.503115 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73f301-327c-4108-ac31-ab8bc7e0fd03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:36.518306 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-b885c3cc-8db3-47b5-bf63-79752042999a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a60ca6fcc27e49bdba767217a0e537c7'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:36.556944 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 895/1797] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:26:36 2022] DELETE /identity/v3/users/a60ca6fcc27e49bdba767217a0e537c7 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:36.562789 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c367a31-2497-4dd5-a436-6fccb1572301 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:36.563041 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c367a31-2497-4dd5-a436-6fccb1572301 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:36.563444 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c367a31-2497-4dd5-a436-6fccb1572301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:36.585219 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-bb73f301-327c-4108-ac31-ab8bc7e0fd03 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:36.588684 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-bb73f301-327c-4108-ac31-ab8bc7e0fd03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:36.662278 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-7c367a31-2497-4dd5-a436-6fccb1572301 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:36.667333 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-7c367a31-2497-4dd5-a436-6fccb1572301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:36.872146 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 903/1798] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:36.922871 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 896/1799] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:37.014519 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb73f301-327c-4108-ac31-ab8bc7e0fd03 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:37.016492 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-bb73f301-327c-4108-ac31-ab8bc7e0fd03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:37.175256 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c367a31-2497-4dd5-a436-6fccb1572301 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:37.177227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-7c367a31-2497-4dd5-a436-6fccb1572301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:37.246781 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb73f301-327c-4108-ac31-ab8bc7e0fd03 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:37.252476 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f38197-2145-49ec-b527-76be9cdae63d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:37.253214 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f38197-2145-49ec-b527-76be9cdae63d service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:37.253488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f38197-2145-49ec-b527-76be9cdae63d service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:37.254207 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f38197-2145-49ec-b527-76be9cdae63d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:37.267331 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f38197-2145-49ec-b527-76be9cdae63d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:37.279841 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f38197-2145-49ec-b527-76be9cdae63d service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:37.300550 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 904/1800] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:37.377623 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c367a31-2497-4dd5-a436-6fccb1572301 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:37.402904 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64226e26-5004-4b5a-a1a4-160cad94dd59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:37.403619 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64226e26-5004-4b5a-a1a4-160cad94dd59 service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:37.403898 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64226e26-5004-4b5a-a1a4-160cad94dd59 service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:37.404171 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64226e26-5004-4b5a-a1a4-160cad94dd59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:37.436398 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:37.450971 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64226e26-5004-4b5a-a1a4-160cad94dd59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:37.450971 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64226e26-5004-4b5a-a1a4-160cad94dd59 service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:37.493871 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 897/1801] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:38.795380 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f38197-2145-49ec-b527-76be9cdae63d service neutron] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:38.796954 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-08f38197-2145-49ec-b527-76be9cdae63d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:38.817885 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f38197-2145-49ec-b527-76be9cdae63d service neutron] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:38.828316 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4373ca6-e171-47f4-96f4-7486c7c71bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:38.829384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4373ca6-e171-47f4-96f4-7486c7c71bba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:38.830366 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4373ca6-e171-47f4-96f4-7486c7c71bba admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:38.830888 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4373ca6-e171-47f4-96f4-7486c7c71bba admin admin] PATH_INFO: `/v3/users/cfe362eac2a143dd82d2bb0688441567` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:38.833911 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4373ca6-e171-47f4-96f4-7486c7c71bba admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfe362eac2a143dd82d2bb0688441567)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:38.864402 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4373ca6-e171-47f4-96f4-7486c7c71bba admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:38.888523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:39.058384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-f4373ca6-e171-47f4-96f4-7486c7c71bba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfe362eac2a143dd82d2bb0688441567'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:39.059359 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-f4373ca6-e171-47f4-96f4-7486c7c71bba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfe362eac2a143dd82d2bb0688441567'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:39.060448 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-f4373ca6-e171-47f4-96f4-7486c7c71bba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfe362eac2a143dd82d2bb0688441567'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:39.085458 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-f4373ca6-e171-47f4-96f4-7486c7c71bba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfe362eac2a143dd82d2bb0688441567'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:39.095186 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-f4373ca6-e171-47f4-96f4-7486c7c71bba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfe362eac2a143dd82d2bb0688441567'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:39.116113 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 905/1802] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:26:38 2022] DELETE /identity/v3/users/cfe362eac2a143dd82d2bb0688441567 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:39.120639 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43306a-ddc7-4b50-ac85-20546a2d7d2f None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:39.121160 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43306a-ddc7-4b50-ac85-20546a2d7d2f None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:39.121238 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43306a-ddc7-4b50-ac85-20546a2d7d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:39.175704 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-ae43306a-ddc7-4b50-ac85-20546a2d7d2f None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:39.180327 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-ae43306a-ddc7-4b50-ac85-20546a2d7d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:39.189015 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4373ca6-e171-47f4-96f4-7486c7c71bba admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:39.190433 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-f4373ca6-e171-47f4-96f4-7486c7c71bba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:39.292650 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4373ca6-e171-47f4-96f4-7486c7c71bba admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:39.297978 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08173c04-47a8-4c33-b149-15ebfdddbe8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:39.298950 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08173c04-47a8-4c33-b149-15ebfdddbe8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:39.299231 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08173c04-47a8-4c33-b149-15ebfdddbe8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:39.299505 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08173c04-47a8-4c33-b149-15ebfdddbe8a admin admin] PATH_INFO: `/v3/projects/c482609d93e840bf8d1be3a03513b91d` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:39.300673 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08173c04-47a8-4c33-b149-15ebfdddbe8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c482609d93e840bf8d1be3a03513b91d)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:39.302607 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08173c04-47a8-4c33-b149-15ebfdddbe8a admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:39.307565 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 898/1803] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:39.332274 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae43306a-ddc7-4b50-ac85-20546a2d7d2f None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:39.336984 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-ae43306a-ddc7-4b50-ac85-20546a2d7d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:39.379980 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-08173c04-47a8-4c33-b149-15ebfdddbe8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c482609d93e840bf8d1be3a03513b91d', 'request_id': 'req-08173c04-47a8-4c33-b149-15ebfdddbe8a'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:39.389377 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-08173c04-47a8-4c33-b149-15ebfdddbe8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c482609d93e840bf8d1be3a03513b91d', 'request_id': 'req-08173c04-47a8-4c33-b149-15ebfdddbe8a'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:39.400641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-08173c04-47a8-4c33-b149-15ebfdddbe8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c482609d93e840bf8d1be3a03513b91d', 'request_id': 'req-08173c04-47a8-4c33-b149-15ebfdddbe8a'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:39.420763 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae43306a-ddc7-4b50-ac85-20546a2d7d2f None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:39.423828 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-08173c04-47a8-4c33-b149-15ebfdddbe8a admin admin] The token cache is being invalidate because project c482609d93e840bf8d1be3a03513b91d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:26:39.424125 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-08173c04-47a8-4c33-b149-15ebfdddbe8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:39.425598 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-08173c04-47a8-4c33-b149-15ebfdddbe8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:39.429559 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 906/1804] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:26:39 2022] DELETE /identity/v3/projects/c482609d93e840bf8d1be3a03513b91d => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:39.433559 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08173c04-47a8-4c33-b149-15ebfdddbe8a admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:39.436810 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-08173c04-47a8-4c33-b149-15ebfdddbe8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:39.464132 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:39.588558 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3ff83d-01c2-43fb-a9c6-ac324e462dc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:39.603793 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ff83d-01c2-43fb-a9c6-ac324e462dc3 service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:39.605071 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ff83d-01c2-43fb-a9c6-ac324e462dc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:39.606640 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ff83d-01c2-43fb-a9c6-ac324e462dc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:39.613233 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-4e3ff83d-01c2-43fb-a9c6-ac324e462dc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:39.638293 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08173c04-47a8-4c33-b149-15ebfdddbe8a admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:39.650023 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09237d-92ae-4642-a4f7-f23fa8623829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:39.651895 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09237d-92ae-4642-a4f7-f23fa8623829 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:39.652207 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09237d-92ae-4642-a4f7-f23fa8623829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:39.652437 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09237d-92ae-4642-a4f7-f23fa8623829 admin admin] PATH_INFO: `/v3/users/8394034b874f409e86dd56986e98da3a` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:39.654734 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb09237d-92ae-4642-a4f7-f23fa8623829 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8394034b874f409e86dd56986e98da3a)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:39.655461 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3ff83d-01c2-43fb-a9c6-ac324e462dc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:39.659373 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3ff83d-01c2-43fb-a9c6-ac324e462dc3 service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:39.662122 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb09237d-92ae-4642-a4f7-f23fa8623829 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:39.700869 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 899/1805] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:39.778898 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3ff83d-01c2-43fb-a9c6-ac324e462dc3 service neutron] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:39.779858 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-4e3ff83d-01c2-43fb-a9c6-ac324e462dc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:39.781942 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-bb09237d-92ae-4642-a4f7-f23fa8623829 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8394034b874f409e86dd56986e98da3a'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:39.783549 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-bb09237d-92ae-4642-a4f7-f23fa8623829 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8394034b874f409e86dd56986e98da3a'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:39.784399 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-bb09237d-92ae-4642-a4f7-f23fa8623829 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8394034b874f409e86dd56986e98da3a'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:39.800083 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3ff83d-01c2-43fb-a9c6-ac324e462dc3 service neutron] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:39.816695 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58401733-f57d-4efd-b083-9c91156bd11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:39.817135 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58401733-f57d-4efd-b083-9c91156bd11e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:39.817414 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58401733-f57d-4efd-b083-9c91156bd11e admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:39.817684 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58401733-f57d-4efd-b083-9c91156bd11e admin admin] PATH_INFO: `/v3/projects/a8c31dbf0edd44c689709c087a51e796` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:39.819456 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-bb09237d-92ae-4642-a4f7-f23fa8623829 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8394034b874f409e86dd56986e98da3a'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:39.830397 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58401733-f57d-4efd-b083-9c91156bd11e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8c31dbf0edd44c689709c087a51e796)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:39.833665 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58401733-f57d-4efd-b083-9c91156bd11e admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:39.840844 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-bb09237d-92ae-4642-a4f7-f23fa8623829 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8394034b874f409e86dd56986e98da3a'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:39.900949 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 907/1806] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:26:39 2022] DELETE /identity/v3/users/8394034b874f409e86dd56986e98da3a => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:39.917081 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-58401733-f57d-4efd-b083-9c91156bd11e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8c31dbf0edd44c689709c087a51e796', 'request_id': 'req-58401733-f57d-4efd-b083-9c91156bd11e'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:39.918421 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:39.927827 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-58401733-f57d-4efd-b083-9c91156bd11e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8c31dbf0edd44c689709c087a51e796', 'request_id': 'req-58401733-f57d-4efd-b083-9c91156bd11e'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:39.937344 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-58401733-f57d-4efd-b083-9c91156bd11e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8c31dbf0edd44c689709c087a51e796', 'request_id': 'req-58401733-f57d-4efd-b083-9c91156bd11e'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:39.955448 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-58401733-f57d-4efd-b083-9c91156bd11e admin admin] The token cache is being invalidate because project a8c31dbf0edd44c689709c087a51e796 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:26:39.956952 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-58401733-f57d-4efd-b083-9c91156bd11e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:39.958182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-58401733-f57d-4efd-b083-9c91156bd11e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:39.961732 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 900/1807] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:26:39 2022] DELETE /identity/v3/projects/a8c31dbf0edd44c689709c087a51e796 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:39.965828 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09237d-92ae-4642-a4f7-f23fa8623829 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:39.967611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-bb09237d-92ae-4642-a4f7-f23fa8623829 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:40.060128 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09237d-92ae-4642-a4f7-f23fa8623829 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:40.089784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e6109-7df4-4bd9-96df-2a2271473bf0 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:40.089784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e6109-7df4-4bd9-96df-2a2271473bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:40.089784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e6109-7df4-4bd9-96df-2a2271473bf0 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:40.090742 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 901/1808] 10.222.0.52 () {54 vars in 921 bytes} [Sat Sep 24 14:26:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 14:26:40.124256 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f1012-4fc3-434f-9db4-86e41aac0af7 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:40.124941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f1012-4fc3-434f-9db4-86e41aac0af7 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:40.125258 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f1012-4fc3-434f-9db4-86e41aac0af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:40.131082 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:40.171108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-960f1012-4fc3-434f-9db4-86e41aac0af7 None None] MFA Rules not processed for user `79eaaae97aac4282a613939e5b768376`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:40.172679 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-960f1012-4fc3-434f-9db4-86e41aac0af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:40.223671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b3596e-61fc-4aec-8cd6-1642800c80b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:40.224698 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3596e-61fc-4aec-8cd6-1642800c80b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:40.225275 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3596e-61fc-4aec-8cd6-1642800c80b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:40.225760 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3596e-61fc-4aec-8cd6-1642800c80b7 admin admin] PATH_INFO: `/v3/users/6b97e389a3344331a52d9c3c7906329a` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:40.227221 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b3596e-61fc-4aec-8cd6-1642800c80b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b97e389a3344331a52d9c3c7906329a)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:40.233198 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b3596e-61fc-4aec-8cd6-1642800c80b7 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:40.367007 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-a4b3596e-61fc-4aec-8cd6-1642800c80b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b97e389a3344331a52d9c3c7906329a'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:40.367721 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-a4b3596e-61fc-4aec-8cd6-1642800c80b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b97e389a3344331a52d9c3c7906329a'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:40.368418 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-a4b3596e-61fc-4aec-8cd6-1642800c80b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b97e389a3344331a52d9c3c7906329a'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:40.386412 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-a4b3596e-61fc-4aec-8cd6-1642800c80b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b97e389a3344331a52d9c3c7906329a'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:40.397462 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-a4b3596e-61fc-4aec-8cd6-1642800c80b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b97e389a3344331a52d9c3c7906329a'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:40.421762 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 908/1809] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:26:39 2022] DELETE /identity/v3/users/6b97e389a3344331a52d9c3c7906329a => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:40.426472 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 902/1810] 10.222.0.52 () {58 vars in 1018 bytes} [Sat Sep 24 14:26:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:26:40.559719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b3596e-61fc-4aec-8cd6-1642800c80b7 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:40.562354 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-a4b3596e-61fc-4aec-8cd6-1642800c80b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:40.675434 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b3596e-61fc-4aec-8cd6-1642800c80b7 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:40.682552 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec50788-3410-47a5-bd53-b3becef14d8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:40.683582 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec50788-3410-47a5-bd53-b3becef14d8f service nova] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:40.683956 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec50788-3410-47a5-bd53-b3becef14d8f service nova] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:40.684751 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec50788-3410-47a5-bd53-b3becef14d8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:40.695466 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec50788-3410-47a5-bd53-b3becef14d8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:40.696942 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec50788-3410-47a5-bd53-b3becef14d8f service nova] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:40.720266 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 909/1811] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:40.806670 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f1012-4fc3-434f-9db4-86e41aac0af7 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:40.808727 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-960f1012-4fc3-434f-9db4-86e41aac0af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:40.833110 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f1012-4fc3-434f-9db4-86e41aac0af7 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:40.856375 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec50788-3410-47a5-bd53-b3becef14d8f service nova] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:40.858298 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-7ec50788-3410-47a5-bd53-b3becef14d8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:40.927575 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:40.943411 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ba5a4-5f86-4fb2-ae5d-fac5be76f6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:40.944817 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ba5a4-5f86-4fb2-ae5d-fac5be76f6b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:40.945475 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ba5a4-5f86-4fb2-ae5d-fac5be76f6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:40.945763 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ba5a4-5f86-4fb2-ae5d-fac5be76f6b3 admin admin] PATH_INFO: `/v3/projects/1b4039e86e054ac7bcbc85087a73e6fc` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:40.947606 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6ba5a4-5f86-4fb2-ae5d-fac5be76f6b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b4039e86e054ac7bcbc85087a73e6fc)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:40.949929 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6ba5a4-5f86-4fb2-ae5d-fac5be76f6b3 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:40.959537 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec50788-3410-47a5-bd53-b3becef14d8f service nova] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:40.964706 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d56a16a-939f-4c76-9e4e-225c9f38ede2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:40.965353 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56a16a-939f-4c76-9e4e-225c9f38ede2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:40.965635 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56a16a-939f-4c76-9e4e-225c9f38ede2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:40.965907 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56a16a-939f-4c76-9e4e-225c9f38ede2 admin admin] PATH_INFO: `/v3/projects/70db651bf80c4b43b5f06f356dc2e539` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:40.967066 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d56a16a-939f-4c76-9e4e-225c9f38ede2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70db651bf80c4b43b5f06f356dc2e539)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:40.968982 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d56a16a-939f-4c76-9e4e-225c9f38ede2 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:41.021244 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-bd6ba5a4-5f86-4fb2-ae5d-fac5be76f6b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b4039e86e054ac7bcbc85087a73e6fc', 'request_id': 'req-bd6ba5a4-5f86-4fb2-ae5d-fac5be76f6b3'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:41.028220 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-bd6ba5a4-5f86-4fb2-ae5d-fac5be76f6b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b4039e86e054ac7bcbc85087a73e6fc', 'request_id': 'req-bd6ba5a4-5f86-4fb2-ae5d-fac5be76f6b3'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:41.035716 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-bd6ba5a4-5f86-4fb2-ae5d-fac5be76f6b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b4039e86e054ac7bcbc85087a73e6fc', 'request_id': 'req-bd6ba5a4-5f86-4fb2-ae5d-fac5be76f6b3'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:41.040210 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-2d56a16a-939f-4c76-9e4e-225c9f38ede2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70db651bf80c4b43b5f06f356dc2e539', 'request_id': 'req-2d56a16a-939f-4c76-9e4e-225c9f38ede2'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:41.047304 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-2d56a16a-939f-4c76-9e4e-225c9f38ede2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70db651bf80c4b43b5f06f356dc2e539', 'request_id': 'req-2d56a16a-939f-4c76-9e4e-225c9f38ede2'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:41.048595 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:41.053086 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-bd6ba5a4-5f86-4fb2-ae5d-fac5be76f6b3 admin admin] The token cache is being invalidate because project 1b4039e86e054ac7bcbc85087a73e6fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:26:41.053907 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-bd6ba5a4-5f86-4fb2-ae5d-fac5be76f6b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:41.055516 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-bd6ba5a4-5f86-4fb2-ae5d-fac5be76f6b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:41.056453 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-2d56a16a-939f-4c76-9e4e-225c9f38ede2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70db651bf80c4b43b5f06f356dc2e539', 'request_id': 'req-2d56a16a-939f-4c76-9e4e-225c9f38ede2'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:41.057913 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 903/1812] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:26:40 2022] DELETE /identity/v3/projects/1b4039e86e054ac7bcbc85087a73e6fc => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:41.058729 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ba5a4-5f86-4fb2-ae5d-fac5be76f6b3 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:41.063646 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-bd6ba5a4-5f86-4fb2-ae5d-fac5be76f6b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:41.074576 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-2d56a16a-939f-4c76-9e4e-225c9f38ede2 admin admin] The token cache is being invalidate because project 70db651bf80c4b43b5f06f356dc2e539 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:26:41.074927 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-2d56a16a-939f-4c76-9e4e-225c9f38ede2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:41.076153 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-2d56a16a-939f-4c76-9e4e-225c9f38ede2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:41.078100 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 910/1813] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:26:40 2022] DELETE /identity/v3/projects/70db651bf80c4b43b5f06f356dc2e539 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:26:41.078916 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d56a16a-939f-4c76-9e4e-225c9f38ede2 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:41.080215 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-2d56a16a-939f-4c76-9e4e-225c9f38ede2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:41.135154 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ba5a4-5f86-4fb2-ae5d-fac5be76f6b3 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:41.147493 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d56a16a-939f-4c76-9e4e-225c9f38ede2 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:41.152162 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c17814-5ede-4dd5-9c32-9ca847179bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:41.152788 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c17814-5ede-4dd5-9c32-9ca847179bb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:41.153065 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c17814-5ede-4dd5-9c32-9ca847179bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:41.153343 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c17814-5ede-4dd5-9c32-9ca847179bb1 admin admin] PATH_INFO: `/v3/users/47bc881aa5e84ead9cd551c0fdd13d15` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:41.154478 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c17814-5ede-4dd5-9c32-9ca847179bb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47bc881aa5e84ead9cd551c0fdd13d15)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:41.156421 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c17814-5ede-4dd5-9c32-9ca847179bb1 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:41.196759 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80b344a-cd36-4e3a-b3dd-439b3ef4641a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:41.197926 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b344a-cd36-4e3a-b3dd-439b3ef4641a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:41.198204 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b344a-cd36-4e3a-b3dd-439b3ef4641a admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:41.198475 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b344a-cd36-4e3a-b3dd-439b3ef4641a admin admin] PATH_INFO: `/v3/projects/4253cdafdba847259586722b26822d68` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:41.199737 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80b344a-cd36-4e3a-b3dd-439b3ef4641a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4253cdafdba847259586722b26822d68)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:41.201661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80b344a-cd36-4e3a-b3dd-439b3ef4641a admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:41.210131 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-72c17814-5ede-4dd5-9c32-9ca847179bb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47bc881aa5e84ead9cd551c0fdd13d15'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:41.210827 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-72c17814-5ede-4dd5-9c32-9ca847179bb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47bc881aa5e84ead9cd551c0fdd13d15'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:41.211455 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-72c17814-5ede-4dd5-9c32-9ca847179bb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47bc881aa5e84ead9cd551c0fdd13d15'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:41.223545 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-72c17814-5ede-4dd5-9c32-9ca847179bb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47bc881aa5e84ead9cd551c0fdd13d15'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:41.229981 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:41.232327 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-72c17814-5ede-4dd5-9c32-9ca847179bb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47bc881aa5e84ead9cd551c0fdd13d15'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:41.246930 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 911/1814] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:26:41 2022] DELETE /identity/v3/users/47bc881aa5e84ead9cd551c0fdd13d15 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:41.248316 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba3275-6a60-4121-b8cf-19f90e4f0362 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:41.248638 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba3275-6a60-4121-b8cf-19f90e4f0362 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:41.248913 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba3275-6a60-4121-b8cf-19f90e4f0362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:41.251602 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-d80b344a-cd36-4e3a-b3dd-439b3ef4641a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4253cdafdba847259586722b26822d68', 'request_id': 'req-d80b344a-cd36-4e3a-b3dd-439b3ef4641a'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:41.258874 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-d80b344a-cd36-4e3a-b3dd-439b3ef4641a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4253cdafdba847259586722b26822d68', 'request_id': 'req-d80b344a-cd36-4e3a-b3dd-439b3ef4641a'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:41.265047 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-d80b344a-cd36-4e3a-b3dd-439b3ef4641a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4253cdafdba847259586722b26822d68', 'request_id': 'req-d80b344a-cd36-4e3a-b3dd-439b3ef4641a'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:41.273958 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-aaba3275-6a60-4121-b8cf-19f90e4f0362 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:41.275237 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-aaba3275-6a60-4121-b8cf-19f90e4f0362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:41.278693 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-d80b344a-cd36-4e3a-b3dd-439b3ef4641a admin admin] The token cache is being invalidate because project 4253cdafdba847259586722b26822d68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:26:41.279017 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-d80b344a-cd36-4e3a-b3dd-439b3ef4641a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:41.279869 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-d80b344a-cd36-4e3a-b3dd-439b3ef4641a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:41.282129 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 904/1815] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:26:41 2022] DELETE /identity/v3/projects/4253cdafdba847259586722b26822d68 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:41.288062 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8561cb-1034-472b-85cc-c3db32866692 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:41.288458 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8561cb-1034-472b-85cc-c3db32866692 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:41.288655 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8561cb-1034-472b-85cc-c3db32866692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:41.315780 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-fc8561cb-1034-472b-85cc-c3db32866692 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:41.316645 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-fc8561cb-1034-472b-85cc-c3db32866692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:41.359657 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 912/1816] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:41.365059 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:41.370099 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c9e88-5b5c-4759-8a27-d6df675794fc None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:41.370468 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c9e88-5b5c-4759-8a27-d6df675794fc None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:41.370863 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c9e88-5b5c-4759-8a27-d6df675794fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:41.391560 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-c91c9e88-5b5c-4759-8a27-d6df675794fc None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:41.392812 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-c91c9e88-5b5c-4759-8a27-d6df675794fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:41.408552 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 905/1817] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:26:41.409305 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 913/1818] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:41.410304 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8561cb-1034-472b-85cc-c3db32866692 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:41.411083 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8f8db-94ab-41ab-8366-0265b17216e6 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:41.411476 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8f8db-94ab-41ab-8366-0265b17216e6 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:41.411622 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8f8db-94ab-41ab-8366-0265b17216e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:41.426352 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8561cb-1034-472b-85cc-c3db32866692 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:41.432699 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d9317-680a-45b8-b01f-3ef69bac4873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:41.433535 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d9317-680a-45b8-b01f-3ef69bac4873 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:41.433817 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d9317-680a-45b8-b01f-3ef69bac4873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:41.434032 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-c4f8f8db-94ab-41ab-8366-0265b17216e6 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:41.434137 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d9317-680a-45b8-b01f-3ef69bac4873 admin admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:41.434719 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6d9317-680a-45b8-b01f-3ef69bac4873 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:26:41.435042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6d9317-680a-45b8-b01f-3ef69bac4873 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:41.435283 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-c4f8f8db-94ab-41ab-8366-0265b17216e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:41.436637 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6d9317-680a-45b8-b01f-3ef69bac4873 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:41.445467 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 914/1819] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:41.450738 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f8f8db-94ab-41ab-8366-0265b17216e6 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:41.452543 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 906/1820] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:26:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:26:41.456647 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d9317-680a-45b8-b01f-3ef69bac4873 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:41.463231 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f8f8db-94ab-41ab-8366-0265b17216e6 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:41.467664 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d9317-680a-45b8-b01f-3ef69bac4873 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:41.468367 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae52670d-cac3-4d97-bf5e-7535626dfa7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:41.468779 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52670d-cac3-4d97-bf5e-7535626dfa7b admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:41.469061 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52670d-cac3-4d97-bf5e-7535626dfa7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:41.469331 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52670d-cac3-4d97-bf5e-7535626dfa7b admin admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:41.470006 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae52670d-cac3-4d97-bf5e-7535626dfa7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:26:41.470312 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae52670d-cac3-4d97-bf5e-7535626dfa7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:41.472006 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae52670d-cac3-4d97-bf5e-7535626dfa7b admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:41.473080 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71a775-25a8-4b5d-8ed1-359ddeb8ef6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:41.473690 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71a775-25a8-4b5d-8ed1-359ddeb8ef6b admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:41.473966 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71a775-25a8-4b5d-8ed1-359ddeb8ef6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:41.474236 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71a775-25a8-4b5d-8ed1-359ddeb8ef6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:41.476087 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad71a775-25a8-4b5d-8ed1-359ddeb8ef6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:41.478270 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad71a775-25a8-4b5d-8ed1-359ddeb8ef6b admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:41.493909 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 915/1821] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:26:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:26:41.499666 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae52670d-cac3-4d97-bf5e-7535626dfa7b admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:41.522679 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae52670d-cac3-4d97-bf5e-7535626dfa7b admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:41.530470 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335bc81e-436f-4b04-b70c-15d75f046342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:41.531125 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335bc81e-436f-4b04-b70c-15d75f046342 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:41.531402 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335bc81e-436f-4b04-b70c-15d75f046342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:41.531677 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335bc81e-436f-4b04-b70c-15d75f046342 admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:41.532313 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335bc81e-436f-4b04-b70c-15d75f046342 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:41.534482 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335bc81e-436f-4b04-b70c-15d75f046342 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:41.536753 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 907/1822] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:26:41 2022] POST /identity/v3/projects => generated 369 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:26:41.543959 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71a775-25a8-4b5d-8ed1-359ddeb8ef6b admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:41.571893 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 916/1823] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:26:41 2022] POST /identity/v3/projects => generated 365 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:26:41.577647 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335bc81e-436f-4b04-b70c-15d75f046342 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:41.664721 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335bc81e-436f-4b04-b70c-15d75f046342 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:41.665029 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71a775-25a8-4b5d-8ed1-359ddeb8ef6b admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:41.670067 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1958159a-76e3-4988-b802-514c03e20c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:41.670689 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1958159a-76e3-4988-b802-514c03e20c12 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:41.671004 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1958159a-76e3-4988-b802-514c03e20c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:41.671275 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1958159a-76e3-4988-b802-514c03e20c12 admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:41.671906 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1958159a-76e3-4988-b802-514c03e20c12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:41.673333 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0999f4-e639-48c3-a037-5227bb603046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:41.673867 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1958159a-76e3-4988-b802-514c03e20c12 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:41.673973 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0999f4-e639-48c3-a037-5227bb603046 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:41.674213 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0999f4-e639-48c3-a037-5227bb603046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:41.674497 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0999f4-e639-48c3-a037-5227bb603046 admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:41.675158 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0999f4-e639-48c3-a037-5227bb603046 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:41.677077 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0999f4-e639-48c3-a037-5227bb603046 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:41.701503 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 917/1824] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:26:41 2022] POST /identity/v3/users => generated 283 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:26:41.702428 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1958159a-76e3-4988-b802-514c03e20c12 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:41.704216 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-1958159a-76e3-4988-b802-514c03e20c12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:41.704744 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 908/1825] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:26:41 2022] POST /identity/v3/users => generated 285 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:26:41.705752 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0999f4-e639-48c3-a037-5227bb603046 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:41.719676 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0999f4-e639-48c3-a037-5227bb603046 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:41.723011 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1958159a-76e3-4988-b802-514c03e20c12 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:41.725731 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa2c60a-e599-477b-b829-70b1ffafd12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:41.726513 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2c60a-e599-477b-b829-70b1ffafd12d admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:41.726995 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2c60a-e599-477b-b829-70b1ffafd12d admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:41.728843 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5141edb3-c30d-41c1-a8ce-0eeec39befca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:41.729677 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5141edb3-c30d-41c1-a8ce-0eeec39befca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:41.729958 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5141edb3-c30d-41c1-a8ce-0eeec39befca admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:41.730227 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5141edb3-c30d-41c1-a8ce-0eeec39befca admin admin] PATH_INFO: `/v3/projects/0035103def854f6b8286a7765b430525` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:41.731655 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5141edb3-c30d-41c1-a8ce-0eeec39befca admin admin] RBAC: Authorizing `identity:delete_project(project_id=0035103def854f6b8286a7765b430525)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:41.733912 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5141edb3-c30d-41c1-a8ce-0eeec39befca admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:41.739952 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2c60a-e599-477b-b829-70b1ffafd12d admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:41.741012 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa2c60a-e599-477b-b829-70b1ffafd12d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:41.744886 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa2c60a-e599-477b-b829-70b1ffafd12d admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:41.775994 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 909/1826] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:41.777079 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa2c60a-e599-477b-b829-70b1ffafd12d admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:41.782011 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:41.795102 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa2c60a-e599-477b-b829-70b1ffafd12d admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:41.805597 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a16bc-7870-49b9-b9d6-3825e2feefa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:41.806394 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a16bc-7870-49b9-b9d6-3825e2feefa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:41.806685 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a16bc-7870-49b9-b9d6-3825e2feefa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:41.807063 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a16bc-7870-49b9-b9d6-3825e2feefa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:41.807809 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03a16bc-7870-49b9-b9d6-3825e2feefa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:41.809438 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03a16bc-7870-49b9-b9d6-3825e2feefa0 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:41.814743 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-5141edb3-c30d-41c1-a8ce-0eeec39befca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0035103def854f6b8286a7765b430525', 'request_id': 'req-5141edb3-c30d-41c1-a8ce-0eeec39befca'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:41.822050 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-5141edb3-c30d-41c1-a8ce-0eeec39befca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0035103def854f6b8286a7765b430525', 'request_id': 'req-5141edb3-c30d-41c1-a8ce-0eeec39befca'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:41.827202 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 910/1827] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:41.828216 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a16bc-7870-49b9-b9d6-3825e2feefa0 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:41.833511 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-5141edb3-c30d-41c1-a8ce-0eeec39befca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0035103def854f6b8286a7765b430525', 'request_id': 'req-5141edb3-c30d-41c1-a8ce-0eeec39befca'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:41.840547 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a16bc-7870-49b9-b9d6-3825e2feefa0 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:41.849520 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-5141edb3-c30d-41c1-a8ce-0eeec39befca admin admin] The token cache is being invalidate because project 0035103def854f6b8286a7765b430525 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:26:41.851070 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-5141edb3-c30d-41c1-a8ce-0eeec39befca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:41.852151 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-5141edb3-c30d-41c1-a8ce-0eeec39befca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:41.853899 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 918/1828] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:26:41 2022] DELETE /identity/v3/projects/0035103def854f6b8286a7765b430525 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:41.858265 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5141edb3-c30d-41c1-a8ce-0eeec39befca admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:41.860464 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-5141edb3-c30d-41c1-a8ce-0eeec39befca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:41.910933 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ca3d5-b6f1-49ac-9e39-59768e9e8537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:41.911610 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ca3d5-b6f1-49ac-9e39-59768e9e8537 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:41.911869 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ca3d5-b6f1-49ac-9e39-59768e9e8537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:41.912138 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ca3d5-b6f1-49ac-9e39-59768e9e8537 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:41.912840 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641ca3d5-b6f1-49ac-9e39-59768e9e8537 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:41.914504 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641ca3d5-b6f1-49ac-9e39-59768e9e8537 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:41.931145 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 911/1829] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:41.931145 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc543f63-5154-4461-a350-91068effc478 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:41.931145 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc543f63-5154-4461-a350-91068effc478 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:41.931145 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc543f63-5154-4461-a350-91068effc478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:41.937568 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5141edb3-c30d-41c1-a8ce-0eeec39befca admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:41.942433 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3d770-1db7-49a8-b162-47d267acd955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:41.943276 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3d770-1db7-49a8-b162-47d267acd955 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:41.943789 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3d770-1db7-49a8-b162-47d267acd955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:41.944103 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3d770-1db7-49a8-b162-47d267acd955 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:41.944507 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:41.944834 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df3d770-1db7-49a8-b162-47d267acd955 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:41.947275 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df3d770-1db7-49a8-b162-47d267acd955 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:41.951786 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-cc543f63-5154-4461-a350-91068effc478 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:41.954109 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-cc543f63-5154-4461-a350-91068effc478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:41.961463 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 919/1830] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:41.961463 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3d770-1db7-49a8-b162-47d267acd955 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:41.961463 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-7df3d770-1db7-49a8-b162-47d267acd955 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:41.962280 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 912/1831] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:41.963081 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc543f63-5154-4461-a350-91068effc478 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:41.972833 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc543f63-5154-4461-a350-91068effc478 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:41.974084 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3d770-1db7-49a8-b162-47d267acd955 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:41.977989 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d5037-7b37-435b-b2a3-06dcfcd8776b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:41.978626 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d5037-7b37-435b-b2a3-06dcfcd8776b admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:41.978938 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d5037-7b37-435b-b2a3-06dcfcd8776b admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:41.979128 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16221745-1ca3-44ec-8c48-b75ed437ad02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:41.979248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d5037-7b37-435b-b2a3-06dcfcd8776b admin admin] PATH_INFO: `/v3/projects/986612297af94de284366296a6fd269f/users/0bbafb8836124f1a849e3168d4c5d194/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:41.979780 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16221745-1ca3-44ec-8c48-b75ed437ad02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:41.980058 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16221745-1ca3-44ec-8c48-b75ed437ad02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:41.980331 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16221745-1ca3-44ec-8c48-b75ed437ad02 admin admin] PATH_INFO: `/v3/projects/652ebf19122e47449d07e6c37a2f7578/users/513f7faab9a041a999478133549a2590/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:41.995030 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634d5037-7b37-435b-b2a3-06dcfcd8776b admin admin] RBAC: Authorizing `identity:create_grant(project_id=986612297af94de284366296a6fd269f, user_id=0bbafb8836124f1a849e3168d4c5d194, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:41.996371 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16221745-1ca3-44ec-8c48-b75ed437ad02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=652ebf19122e47449d07e6c37a2f7578, user_id=513f7faab9a041a999478133549a2590, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:41.997710 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634d5037-7b37-435b-b2a3-06dcfcd8776b admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:41.999136 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16221745-1ca3-44ec-8c48-b75ed437ad02 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:42.010783 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 913/1832] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:41 2022] PUT /identity/v3/projects/986612297af94de284366296a6fd269f/users/0bbafb8836124f1a849e3168d4c5d194/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:42.020759 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee92d7-811c-4550-8728-be627937901e None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:42.020759 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee92d7-811c-4550-8728-be627937901e None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:42.020759 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee92d7-811c-4550-8728-be627937901e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:42.021027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 920/1833] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:41 2022] PUT /identity/v3/projects/652ebf19122e47449d07e6c37a2f7578/users/513f7faab9a041a999478133549a2590/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:42.022066 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c1724-b609-41a5-875a-b5e6036e4649 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:42.022354 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c1724-b609-41a5-875a-b5e6036e4649 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:42.022632 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c1724-b609-41a5-875a-b5e6036e4649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:42.035160 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-cfee92d7-811c-4550-8728-be627937901e None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:42.036376 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-cfee92d7-811c-4550-8728-be627937901e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:42.052756 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-0d1c1724-b609-41a5-875a-b5e6036e4649 None None] MFA Rules not processed for user `0bbafb8836124f1a849e3168d4c5d194`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:42.054588 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-0d1c1724-b609-41a5-875a-b5e6036e4649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:42.104188 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 914/1834] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:42.105637 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463db9c-38cf-4693-ae5b-ddc117999b8c None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:42.105924 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463db9c-38cf-4693-ae5b-ddc117999b8c None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:42.106197 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463db9c-38cf-4693-ae5b-ddc117999b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:42.133584 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 921/1835] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:26:42.134584 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c1724-b609-41a5-875a-b5e6036e4649 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:42.138727 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-1463db9c-38cf-4693-ae5b-ddc117999b8c None None] MFA Rules not processed for user `513f7faab9a041a999478133549a2590`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:42.144363 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-1463db9c-38cf-4693-ae5b-ddc117999b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:42.145648 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c1724-b609-41a5-875a-b5e6036e4649 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:42.150592 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61b98e-0c4b-408f-b6c6-61e249d816de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:42.151168 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61b98e-0c4b-408f-b6c6-61e249d816de admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:42.151449 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61b98e-0c4b-408f-b6c6-61e249d816de admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:42.151718 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61b98e-0c4b-408f-b6c6-61e249d816de admin admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:42.152364 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec61b98e-0c4b-408f-b6c6-61e249d816de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:26:42.152660 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec61b98e-0c4b-408f-b6c6-61e249d816de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:42.154871 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec61b98e-0c4b-408f-b6c6-61e249d816de admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:42.171889 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 922/1836] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:26:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:26:42.171889 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61b98e-0c4b-408f-b6c6-61e249d816de admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:42.176785 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:42.183432 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61b98e-0c4b-408f-b6c6-61e249d816de admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:42.190392 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facb56a1-d535-4515-84a7-4a46ae1436ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:42.191542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb56a1-d535-4515-84a7-4a46ae1436ef admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:42.192592 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb56a1-d535-4515-84a7-4a46ae1436ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:42.193282 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb56a1-d535-4515-84a7-4a46ae1436ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:42.194270 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facb56a1-d535-4515-84a7-4a46ae1436ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:42.196352 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facb56a1-d535-4515-84a7-4a46ae1436ef admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:42.226662 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 915/1837] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:42.227633 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1463db9c-38cf-4693-ae5b-ddc117999b8c None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:42.232283 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 923/1838] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:26:42 2022] POST /identity/v3/projects => generated 365 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:42.233290 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facb56a1-d535-4515-84a7-4a46ae1436ef admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:42.237924 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1463db9c-38cf-4693-ae5b-ddc117999b8c None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:42.301777 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facb56a1-d535-4515-84a7-4a46ae1436ef admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:42.306454 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ec6060-a6de-4eea-9808-2db6ed7499fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:42.307100 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec6060-a6de-4eea-9808-2db6ed7499fe admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:42.307241 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae26753-59e9-4c9c-9265-b4c2cd61eb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:42.307896 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec6060-a6de-4eea-9808-2db6ed7499fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:42.307896 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec6060-a6de-4eea-9808-2db6ed7499fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:42.308136 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae26753-59e9-4c9c-9265-b4c2cd61eb77 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:42.308136 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae26753-59e9-4c9c-9265-b4c2cd61eb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:42.308300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae26753-59e9-4c9c-9265-b4c2cd61eb77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:42.308388 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ec6060-a6de-4eea-9808-2db6ed7499fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:42.308857 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae26753-59e9-4c9c-9265-b4c2cd61eb77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:42.310165 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ec6060-a6de-4eea-9808-2db6ed7499fe admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:42.310769 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae26753-59e9-4c9c-9265-b4c2cd61eb77 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:42.321685 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:42.339517 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 924/1839] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:26:42 2022] POST /identity/v3/projects => generated 363 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:42.340079 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 916/1840] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:26:42 2022] POST /identity/v3/projects => generated 387 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:42.340418 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae26753-59e9-4c9c-9265-b4c2cd61eb77 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:42.346872 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ec6060-a6de-4eea-9808-2db6ed7499fe admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:42.407605 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae26753-59e9-4c9c-9265-b4c2cd61eb77 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:42.420984 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ec6060-a6de-4eea-9808-2db6ed7499fe admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:42.425665 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d72b6-f1ea-4bed-96f2-5c412db1edd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:42.426279 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d72b6-f1ea-4bed-96f2-5c412db1edd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:42.426742 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d72b6-f1ea-4bed-96f2-5c412db1edd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:42.427041 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d72b6-f1ea-4bed-96f2-5c412db1edd6 admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:42.427674 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20d72b6-f1ea-4bed-96f2-5c412db1edd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:42.429656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20d72b6-f1ea-4bed-96f2-5c412db1edd6 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:42.453728 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 917/1841] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:26:42 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:42.453728 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d72b6-f1ea-4bed-96f2-5c412db1edd6 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:42.461864 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d72b6-f1ea-4bed-96f2-5c412db1edd6 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:42.466399 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b427567-a2ca-4407-b9ac-9a86e3b7e553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:42.467006 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b427567-a2ca-4407-b9ac-9a86e3b7e553 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:42.467289 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b427567-a2ca-4407-b9ac-9a86e3b7e553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:42.467560 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b427567-a2ca-4407-b9ac-9a86e3b7e553 admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:42.468177 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b427567-a2ca-4407-b9ac-9a86e3b7e553 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:42.468727 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3764b742-72aa-481b-8974-66d41152ae39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:42.469334 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3764b742-72aa-481b-8974-66d41152ae39 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:42.469611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3764b742-72aa-481b-8974-66d41152ae39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:42.469882 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3764b742-72aa-481b-8974-66d41152ae39 admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:42.470072 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b427567-a2ca-4407-b9ac-9a86e3b7e553 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:42.470503 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3764b742-72aa-481b-8974-66d41152ae39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:42.472439 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3764b742-72aa-481b-8974-66d41152ae39 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:42.491454 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 918/1842] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:26:42 2022] POST /identity/v3/users => generated 294 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:42.492262 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b427567-a2ca-4407-b9ac-9a86e3b7e553 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:42.496984 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 925/1843] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:26:42 2022] POST /identity/v3/users => generated 289 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:42.497935 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3764b742-72aa-481b-8974-66d41152ae39 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:42.503114 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:42.504503 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b427567-a2ca-4407-b9ac-9a86e3b7e553 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:42.509186 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3764b742-72aa-481b-8974-66d41152ae39 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:42.509390 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c001f69-b599-48b2-a002-50ed230ebaae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:42.510016 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c001f69-b599-48b2-a002-50ed230ebaae admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:42.510321 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c001f69-b599-48b2-a002-50ed230ebaae admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:42.510624 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c001f69-b599-48b2-a002-50ed230ebaae admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:42.511439 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c001f69-b599-48b2-a002-50ed230ebaae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:42.513039 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c001f69-b599-48b2-a002-50ed230ebaae admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:42.514879 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff28023-52c1-4015-8400-c95d7c242f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:42.515503 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff28023-52c1-4015-8400-c95d7c242f2a admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:42.515782 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff28023-52c1-4015-8400-c95d7c242f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:42.516085 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff28023-52c1-4015-8400-c95d7c242f2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:42.516722 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff28023-52c1-4015-8400-c95d7c242f2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:42.518287 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff28023-52c1-4015-8400-c95d7c242f2a admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:42.524355 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 919/1844] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:42.525166 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c001f69-b599-48b2-a002-50ed230ebaae admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:42.536675 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 926/1845] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:42.536675 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff28023-52c1-4015-8400-c95d7c242f2a admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:42.536935 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c001f69-b599-48b2-a002-50ed230ebaae admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:42.541306 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2fb650-2271-4831-b220-1201457d4983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:42.541928 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2fb650-2271-4831-b220-1201457d4983 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:42.542199 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2fb650-2271-4831-b220-1201457d4983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:42.542466 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2fb650-2271-4831-b220-1201457d4983 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:42.543122 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2fb650-2271-4831-b220-1201457d4983 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:42.544697 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2fb650-2271-4831-b220-1201457d4983 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:42.547071 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff28023-52c1-4015-8400-c95d7c242f2a admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:42.551803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71e1105-916f-457e-8135-f27980e88643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:42.552441 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e1105-916f-457e-8135-f27980e88643 admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:42.552714 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e1105-916f-457e-8135-f27980e88643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:42.552990 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e1105-916f-457e-8135-f27980e88643 admin admin] PATH_INFO: `/v3/projects/d25245b7fe7e4beba7f9e19bbca30b19/users/33558a40a4984a00bea603e5ae8d68d6/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:42.557857 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 920/1846] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:42.557857 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2fb650-2271-4831-b220-1201457d4983 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:42.565337 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2fb650-2271-4831-b220-1201457d4983 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:42.567925 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71e1105-916f-457e-8135-f27980e88643 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d25245b7fe7e4beba7f9e19bbca30b19, user_id=33558a40a4984a00bea603e5ae8d68d6, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:42.570251 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f1cdd6-4d8d-421e-8918-16c64558e065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:42.570587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71e1105-916f-457e-8135-f27980e88643 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:42.570903 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1cdd6-4d8d-421e-8918-16c64558e065 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:42.571211 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1cdd6-4d8d-421e-8918-16c64558e065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:42.571479 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1cdd6-4d8d-421e-8918-16c64558e065 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:42.572104 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f1cdd6-4d8d-421e-8918-16c64558e065 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:42.573725 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f1cdd6-4d8d-421e-8918-16c64558e065 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:42.587936 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 927/1847] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:42 2022] PUT /identity/v3/projects/d25245b7fe7e4beba7f9e19bbca30b19/users/33558a40a4984a00bea603e5ae8d68d6/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:42.587936 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71e1105-916f-457e-8135-f27980e88643 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:42.590093 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 921/1848] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:42.591219 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f1cdd6-4d8d-421e-8918-16c64558e065 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:42.602645 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f1cdd6-4d8d-421e-8918-16c64558e065 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:42.648071 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71e1105-916f-457e-8135-f27980e88643 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:42.652869 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b325f-8f39-44c3-9abc-27adefe5fcad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:42.653524 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b325f-8f39-44c3-9abc-27adefe5fcad admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:42.653820 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b325f-8f39-44c3-9abc-27adefe5fcad admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:42.654058 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b325f-8f39-44c3-9abc-27adefe5fcad admin admin] PATH_INFO: `/v3/projects/6db5619ed1c1414985e51d4faed65e75/users/c1c8120fb76c4dca9bbf78cfee16a76b/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:42.661030 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f73b295-81cd-4e43-863d-2cb93ccde501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:42.661638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73b295-81cd-4e43-863d-2cb93ccde501 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:42.661914 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73b295-81cd-4e43-863d-2cb93ccde501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:42.662183 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73b295-81cd-4e43-863d-2cb93ccde501 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:42.663401 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f73b295-81cd-4e43-863d-2cb93ccde501 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:42.665310 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f73b295-81cd-4e43-863d-2cb93ccde501 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:42.670465 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8b325f-8f39-44c3-9abc-27adefe5fcad admin admin] RBAC: Authorizing `identity:create_grant(project_id=6db5619ed1c1414985e51d4faed65e75, user_id=c1c8120fb76c4dca9bbf78cfee16a76b, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:42.673152 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8b325f-8f39-44c3-9abc-27adefe5fcad admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:42.676314 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 922/1849] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:42.677113 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f73b295-81cd-4e43-863d-2cb93ccde501 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:42.689467 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f73b295-81cd-4e43-863d-2cb93ccde501 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:42.695571 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 928/1850] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:42 2022] PUT /identity/v3/projects/6db5619ed1c1414985e51d4faed65e75/users/c1c8120fb76c4dca9bbf78cfee16a76b/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:42.695571 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b325f-8f39-44c3-9abc-27adefe5fcad admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:42.721154 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:42.760493 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fca662-bfe8-4f7a-9f22-e3a73f56c94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:42.761294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fca662-bfe8-4f7a-9f22-e3a73f56c94f admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:42.761579 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fca662-bfe8-4f7a-9f22-e3a73f56c94f admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:42.761852 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fca662-bfe8-4f7a-9f22-e3a73f56c94f admin admin] PATH_INFO: `/v3/projects/b1b818cca5494181a9de584f0e3f0d2a/users/ee5515fa23d24b08a9e6458d8f7b2e81/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:42.762844 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b325f-8f39-44c3-9abc-27adefe5fcad admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:42.769118 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7c921-1297-43b1-8046-25d0d7f9cc31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:42.770127 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7c921-1297-43b1-8046-25d0d7f9cc31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:42.770637 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7c921-1297-43b1-8046-25d0d7f9cc31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:42.771009 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7c921-1297-43b1-8046-25d0d7f9cc31 admin admin] PATH_INFO: `/v3/projects/d25245b7fe7e4beba7f9e19bbca30b19/users/33558a40a4984a00bea603e5ae8d68d6/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:42.773955 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f7c921-1297-43b1-8046-25d0d7f9cc31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d25245b7fe7e4beba7f9e19bbca30b19, user_id=33558a40a4984a00bea603e5ae8d68d6, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:42.776854 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f7c921-1297-43b1-8046-25d0d7f9cc31 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:42.788961 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fca662-bfe8-4f7a-9f22-e3a73f56c94f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1b818cca5494181a9de584f0e3f0d2a, user_id=ee5515fa23d24b08a9e6458d8f7b2e81, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:42.791406 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 929/1851] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:42 2022] PUT /identity/v3/projects/d25245b7fe7e4beba7f9e19bbca30b19/users/33558a40a4984a00bea603e5ae8d68d6/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:42.792484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7c921-1297-43b1-8046-25d0d7f9cc31 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:42.795414 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fca662-bfe8-4f7a-9f22-e3a73f56c94f admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:42.819552 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 923/1852] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:42 2022] PUT /identity/v3/projects/b1b818cca5494181a9de584f0e3f0d2a/users/ee5515fa23d24b08a9e6458d8f7b2e81/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:42.821162 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f920676-28f0-462d-b018-dd4178581264 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:42.821897 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f920676-28f0-462d-b018-dd4178581264 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:42.822387 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f920676-28f0-462d-b018-dd4178581264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:42.856318 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-1f920676-28f0-462d-b018-dd4178581264 None None] MFA Rules not processed for user `33558a40a4984a00bea603e5ae8d68d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:42.857615 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-1f920676-28f0-462d-b018-dd4178581264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:42.894433 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:42.926404 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7c921-1297-43b1-8046-25d0d7f9cc31 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:42.956432 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 924/1853] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:42 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:42.957966 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190c788-c79f-4ab1-b58c-0e93df70a323 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:42.958249 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190c788-c79f-4ab1-b58c-0e93df70a323 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:42.958517 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190c788-c79f-4ab1-b58c-0e93df70a323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:43.002672 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-6190c788-c79f-4ab1-b58c-0e93df70a323 None None] MFA Rules not processed for user `ee5515fa23d24b08a9e6458d8f7b2e81`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:43.004326 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-6190c788-c79f-4ab1-b58c-0e93df70a323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:43.029481 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a393fbbb-4fb7-4e0a-9ca3-bee1f400ef51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:43.030099 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393fbbb-4fb7-4e0a-9ca3-bee1f400ef51 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:43.030383 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393fbbb-4fb7-4e0a-9ca3-bee1f400ef51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:43.030655 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393fbbb-4fb7-4e0a-9ca3-bee1f400ef51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:43.031355 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a393fbbb-4fb7-4e0a-9ca3-bee1f400ef51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:43.033077 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a393fbbb-4fb7-4e0a-9ca3-bee1f400ef51 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:43.043646 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 930/1854] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:43.044627 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a393fbbb-4fb7-4e0a-9ca3-bee1f400ef51 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:43.046408 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-a393fbbb-4fb7-4e0a-9ca3-bee1f400ef51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:43.081990 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 925/1855] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:42 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:43.082861 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6190c788-c79f-4ab1-b58c-0e93df70a323 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:43.094339 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6190c788-c79f-4ab1-b58c-0e93df70a323 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:43.099738 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e819279-d733-43eb-b4a4-530d78f39b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:43.099738 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e819279-d733-43eb-b4a4-530d78f39b07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:43.100045 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e819279-d733-43eb-b4a4-530d78f39b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:43.100283 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e819279-d733-43eb-b4a4-530d78f39b07 admin admin] PATH_INFO: `/v3/projects/6db5619ed1c1414985e51d4faed65e75/users/c1c8120fb76c4dca9bbf78cfee16a76b/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:43.102222 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e819279-d733-43eb-b4a4-530d78f39b07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6db5619ed1c1414985e51d4faed65e75, user_id=c1c8120fb76c4dca9bbf78cfee16a76b, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:43.104899 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e819279-d733-43eb-b4a4-530d78f39b07 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:43.111398 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a393fbbb-4fb7-4e0a-9ca3-bee1f400ef51 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:43.116215 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 926/1856] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:43 2022] PUT /identity/v3/projects/6db5619ed1c1414985e51d4faed65e75/users/c1c8120fb76c4dca9bbf78cfee16a76b/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:43.117168 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e819279-d733-43eb-b4a4-530d78f39b07 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:43.130668 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:43.173507 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71b9f6-24a8-4c38-ae39-1d17fedfdf00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:43.174040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71b9f6-24a8-4c38-ae39-1d17fedfdf00 service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:43.174319 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71b9f6-24a8-4c38-ae39-1d17fedfdf00 service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:43.174847 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71b9f6-24a8-4c38-ae39-1d17fedfdf00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:43.187555 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e819279-d733-43eb-b4a4-530d78f39b07 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:43.198031 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b981409-2e6c-4703-ac0d-aecfdc2ee53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:43.198679 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b981409-2e6c-4703-ac0d-aecfdc2ee53d admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:43.199004 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b981409-2e6c-4703-ac0d-aecfdc2ee53d admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:43.199280 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b981409-2e6c-4703-ac0d-aecfdc2ee53d admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:43.199958 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b981409-2e6c-4703-ac0d-aecfdc2ee53d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:43.201887 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b981409-2e6c-4703-ac0d-aecfdc2ee53d admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:43.238067 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 927/1857] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:26:43 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:43.239782 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d0390-1a29-4e72-bac5-0fdc473f700c None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:43.240066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d0390-1a29-4e72-bac5-0fdc473f700c None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:43.240337 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d0390-1a29-4e72-bac5-0fdc473f700c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:43.245488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:43.249736 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e71b9f6-24a8-4c38-ae39-1d17fedfdf00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:43.251529 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e71b9f6-24a8-4c38-ae39-1d17fedfdf00 service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:43.263686 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 931/1858] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:43 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:43.264700 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71b9f6-24a8-4c38-ae39-1d17fedfdf00 service cinder] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:43.270836 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-da0d0390-1a29-4e72-bac5-0fdc473f700c None None] MFA Rules not processed for user `c1c8120fb76c4dca9bbf78cfee16a76b`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:43.272132 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-da0d0390-1a29-4e72-bac5-0fdc473f700c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:43.277101 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71b9f6-24a8-4c38-ae39-1d17fedfdf00 service cinder] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:43.360206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b5730-23be-46b8-bfd4-31cacae1cc31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:43.361128 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b5730-23be-46b8-bfd4-31cacae1cc31 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:43.361469 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b5730-23be-46b8-bfd4-31cacae1cc31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:43.361821 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b5730-23be-46b8-bfd4-31cacae1cc31 admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:43.362538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b5730-23be-46b8-bfd4-31cacae1cc31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:43.364589 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b5730-23be-46b8-bfd4-31cacae1cc31 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:43.391706 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 928/1859] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:43 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:43.393568 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0d0390-1a29-4e72-bac5-0fdc473f700c None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:43.400976 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 932/1860] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:26:43 2022] POST /identity/v3/users => generated 299 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:43.408062 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b5730-23be-46b8-bfd4-31cacae1cc31 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:43.421745 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b5730-23be-46b8-bfd4-31cacae1cc31 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:43.428653 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440fd5e-3fdc-4cb9-96db-c13cede2b2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:43.429436 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440fd5e-3fdc-4cb9-96db-c13cede2b2e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:43.430003 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440fd5e-3fdc-4cb9-96db-c13cede2b2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:43.430422 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440fd5e-3fdc-4cb9-96db-c13cede2b2e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:43.431245 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a440fd5e-3fdc-4cb9-96db-c13cede2b2e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:43.434036 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a440fd5e-3fdc-4cb9-96db-c13cede2b2e5 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:43.445850 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 933/1861] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:43.446655 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440fd5e-3fdc-4cb9-96db-c13cede2b2e5 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:43.448296 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-a440fd5e-3fdc-4cb9-96db-c13cede2b2e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:43.463806 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0d0390-1a29-4e72-bac5-0fdc473f700c None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:43.467642 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07811ca1-7f0c-405f-8682-df0b0a1457f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:43.468242 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07811ca1-7f0c-405f-8682-df0b0a1457f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:43.468518 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07811ca1-7f0c-405f-8682-df0b0a1457f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:43.468793 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07811ca1-7f0c-405f-8682-df0b0a1457f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:43.503621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:43.504460 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440fd5e-3fdc-4cb9-96db-c13cede2b2e5 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:43.508580 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd900cd-3dbb-4512-a006-c4cefb25f06c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:43.509132 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd900cd-3dbb-4512-a006-c4cefb25f06c service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:43.509908 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd900cd-3dbb-4512-a006-c4cefb25f06c service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:43.510284 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd900cd-3dbb-4512-a006-c4cefb25f06c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:43.520276 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd900cd-3dbb-4512-a006-c4cefb25f06c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:43.521889 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd900cd-3dbb-4512-a006-c4cefb25f06c service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:43.531087 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07811ca1-7f0c-405f-8682-df0b0a1457f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:43.531329 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 934/1862] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:43 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:43.532011 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd900cd-3dbb-4512-a006-c4cefb25f06c service cinder] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:43.532542 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07811ca1-7f0c-405f-8682-df0b0a1457f5 service cinder] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:43.542505 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd900cd-3dbb-4512-a006-c4cefb25f06c service cinder] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:43.543083 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 929/1863] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:43.548701 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4dd0d2-9ded-42b6-9dd1-b025fbe3abd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:43.549382 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4dd0d2-9ded-42b6-9dd1-b025fbe3abd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:43.549657 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4dd0d2-9ded-42b6-9dd1-b025fbe3abd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:43.549929 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4dd0d2-9ded-42b6-9dd1-b025fbe3abd2 admin admin] PATH_INFO: `/v3/projects/251046b70b0045148ac2cb70cd10581b/users/882f9799152040e394ac51eb56e8c14b/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:43.570315 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4dd0d2-9ded-42b6-9dd1-b025fbe3abd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=251046b70b0045148ac2cb70cd10581b, user_id=882f9799152040e394ac51eb56e8c14b, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:43.572947 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4dd0d2-9ded-42b6-9dd1-b025fbe3abd2 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:43.583688 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07811ca1-7f0c-405f-8682-df0b0a1457f5 service cinder] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:43.589564 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 935/1864] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:43 2022] PUT /identity/v3/projects/251046b70b0045148ac2cb70cd10581b/users/882f9799152040e394ac51eb56e8c14b/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:43.594017 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07811ca1-7f0c-405f-8682-df0b0a1457f5 service cinder] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:43.595374 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4dd0d2-9ded-42b6-9dd1-b025fbe3abd2 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:43.661392 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66437db9-ed6a-4c65-97da-0ff1f42364eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:43.661993 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:43.662121 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66437db9-ed6a-4c65-97da-0ff1f42364eb service cinder] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:43.662265 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66437db9-ed6a-4c65-97da-0ff1f42364eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:43.663084 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66437db9-ed6a-4c65-97da-0ff1f42364eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:43.680744 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4dd0d2-9ded-42b6-9dd1-b025fbe3abd2 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:43.686549 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811a683c-95bf-4d16-ae20-6b21b96f5aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:43.687079 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a683c-95bf-4d16-ae20-6b21b96f5aef admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:43.687363 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a683c-95bf-4d16-ae20-6b21b96f5aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:43.687637 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a683c-95bf-4d16-ae20-6b21b96f5aef admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:43.688289 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811a683c-95bf-4d16-ae20-6b21b96f5aef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:43.690956 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811a683c-95bf-4d16-ae20-6b21b96f5aef admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:43.705773 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 936/1865] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:43 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 14:26:43.706716 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811a683c-95bf-4d16-ae20-6b21b96f5aef admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:43.717778 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811a683c-95bf-4d16-ae20-6b21b96f5aef admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:43.722847 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca611bf-4a6c-4cdb-92cd-b9f0295d0070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:43.724014 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca611bf-4a6c-4cdb-92cd-b9f0295d0070 admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:43.724014 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca611bf-4a6c-4cdb-92cd-b9f0295d0070 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:43.724424 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca611bf-4a6c-4cdb-92cd-b9f0295d0070 admin admin] PATH_INFO: `/v3/projects/251046b70b0045148ac2cb70cd10581b/users/882f9799152040e394ac51eb56e8c14b/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:43.727035 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca611bf-4a6c-4cdb-92cd-b9f0295d0070 admin admin] RBAC: Authorizing `identity:create_grant(project_id=251046b70b0045148ac2cb70cd10581b, user_id=882f9799152040e394ac51eb56e8c14b, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:43.729738 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca611bf-4a6c-4cdb-92cd-b9f0295d0070 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:43.732989 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66437db9-ed6a-4c65-97da-0ff1f42364eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:43.734504 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66437db9-ed6a-4c65-97da-0ff1f42364eb service cinder] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:43.740137 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 937/1866] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:43 2022] PUT /identity/v3/projects/251046b70b0045148ac2cb70cd10581b/users/882f9799152040e394ac51eb56e8c14b/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:43.744128 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 930/1867] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:43 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 14:26:43.752128 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b5b93-e4b1-462b-928d-df2cdedbe076 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:43.752398 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b5b93-e4b1-462b-928d-df2cdedbe076 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:43.752662 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b5b93-e4b1-462b-928d-df2cdedbe076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:43.780352 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-5d2b5b93-e4b1-462b-928d-df2cdedbe076 None None] MFA Rules not processed for user `882f9799152040e394ac51eb56e8c14b`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:43.781556 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-5d2b5b93-e4b1-462b-928d-df2cdedbe076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:43.861966 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 938/1868] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:43 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:43.895663 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66437db9-ed6a-4c65-97da-0ff1f42364eb service cinder] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:43.910753 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66437db9-ed6a-4c65-97da-0ff1f42364eb service cinder] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:43.965456 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a42fe-1832-4ad8-8abd-a451c8777760 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:43.966098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a42fe-1832-4ad8-8abd-a451c8777760 service cinder] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:43.966529 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a42fe-1832-4ad8-8abd-a451c8777760 service cinder] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:43.966806 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a42fe-1832-4ad8-8abd-a451c8777760 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:43.977459 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37a42fe-1832-4ad8-8abd-a451c8777760 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:43.979396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37a42fe-1832-4ad8-8abd-a451c8777760 service cinder] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:43.990353 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 931/1869] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:43 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:44.024954 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2b5b93-e4b1-462b-928d-df2cdedbe076 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:44.093027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2b5b93-e4b1-462b-928d-df2cdedbe076 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:44.097716 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5095dd58-1797-4147-9ff5-82a4cc1558db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:44.098465 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095dd58-1797-4147-9ff5-82a4cc1558db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:44.098637 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095dd58-1797-4147-9ff5-82a4cc1558db admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:44.098924 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095dd58-1797-4147-9ff5-82a4cc1558db admin admin] PATH_INFO: `/v3/users/ee5515fa23d24b08a9e6458d8f7b2e81` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:44.100189 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5095dd58-1797-4147-9ff5-82a4cc1558db admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee5515fa23d24b08a9e6458d8f7b2e81)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:44.102208 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5095dd58-1797-4147-9ff5-82a4cc1558db admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:44.169342 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-5095dd58-1797-4147-9ff5-82a4cc1558db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee5515fa23d24b08a9e6458d8f7b2e81'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:44.170069 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-5095dd58-1797-4147-9ff5-82a4cc1558db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee5515fa23d24b08a9e6458d8f7b2e81'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:44.170773 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-5095dd58-1797-4147-9ff5-82a4cc1558db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee5515fa23d24b08a9e6458d8f7b2e81'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:44.182163 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:44.184372 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-5095dd58-1797-4147-9ff5-82a4cc1558db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee5515fa23d24b08a9e6458d8f7b2e81'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:44.193581 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-5095dd58-1797-4147-9ff5-82a4cc1558db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee5515fa23d24b08a9e6458d8f7b2e81'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:44.209611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 939/1870] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:26:44 2022] DELETE /identity/v3/users/ee5515fa23d24b08a9e6458d8f7b2e81 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:44.213917 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd2cdd-d68e-4ce0-92d0-8762c29225b3 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:44.214646 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd2cdd-d68e-4ce0-92d0-8762c29225b3 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:44.214646 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd2cdd-d68e-4ce0-92d0-8762c29225b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:44.234418 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-8ecd2cdd-d68e-4ce0-92d0-8762c29225b3 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:44.235798 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-8ecd2cdd-d68e-4ce0-92d0-8762c29225b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:44.266515 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:44.300418 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 932/1871] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:26:44.310758 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5095dd58-1797-4147-9ff5-82a4cc1558db admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:44.312198 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-5095dd58-1797-4147-9ff5-82a4cc1558db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:44.373874 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5095dd58-1797-4147-9ff5-82a4cc1558db admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:44.377501 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e6ed7-7d41-4788-8bd5-f508f8665407 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:44.378113 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e6ed7-7d41-4788-8bd5-f508f8665407 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:44.378392 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e6ed7-7d41-4788-8bd5-f508f8665407 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:44.378665 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e6ed7-7d41-4788-8bd5-f508f8665407 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:44.388747 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151e6ed7-7d41-4788-8bd5-f508f8665407 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:44.390177 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151e6ed7-7d41-4788-8bd5-f508f8665407 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:44.399810 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 940/1872] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:44.742262 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd2cdd-d68e-4ce0-92d0-8762c29225b3 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:44.742262 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-8ecd2cdd-d68e-4ce0-92d0-8762c29225b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:44.757829 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd2cdd-d68e-4ce0-92d0-8762c29225b3 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:44.762768 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db66e397-7915-4400-a8fd-9e68b43ba927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:44.763450 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66e397-7915-4400-a8fd-9e68b43ba927 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:44.763730 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66e397-7915-4400-a8fd-9e68b43ba927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:44.764003 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66e397-7915-4400-a8fd-9e68b43ba927 admin admin] PATH_INFO: `/v3/projects/b1b818cca5494181a9de584f0e3f0d2a` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:44.765145 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db66e397-7915-4400-a8fd-9e68b43ba927 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1b818cca5494181a9de584f0e3f0d2a)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:44.767086 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db66e397-7915-4400-a8fd-9e68b43ba927 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:44.839729 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-db66e397-7915-4400-a8fd-9e68b43ba927 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1b818cca5494181a9de584f0e3f0d2a', 'request_id': 'req-db66e397-7915-4400-a8fd-9e68b43ba927'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:44.851757 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-db66e397-7915-4400-a8fd-9e68b43ba927 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1b818cca5494181a9de584f0e3f0d2a', 'request_id': 'req-db66e397-7915-4400-a8fd-9e68b43ba927'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:44.861705 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-db66e397-7915-4400-a8fd-9e68b43ba927 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1b818cca5494181a9de584f0e3f0d2a', 'request_id': 'req-db66e397-7915-4400-a8fd-9e68b43ba927'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:44.897466 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-db66e397-7915-4400-a8fd-9e68b43ba927 admin admin] The token cache is being invalidate because project b1b818cca5494181a9de584f0e3f0d2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:26:44.897466 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-db66e397-7915-4400-a8fd-9e68b43ba927 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:44.912256 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-db66e397-7915-4400-a8fd-9e68b43ba927 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:44.934093 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 933/1873] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:26:44 2022] DELETE /identity/v3/projects/b1b818cca5494181a9de584f0e3f0d2a => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:44.938483 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e6ed7-7d41-4788-8bd5-f508f8665407 service neutron] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:44.976837 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e6ed7-7d41-4788-8bd5-f508f8665407 service neutron] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:45.171327 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1830d61b-f50d-49f4-b245-51fba3190ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:45.171926 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830d61b-f50d-49f4-b245-51fba3190ff0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:45.172034 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830d61b-f50d-49f4-b245-51fba3190ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:45.172305 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830d61b-f50d-49f4-b245-51fba3190ff0 admin admin] PATH_INFO: `/v3/users/882f9799152040e394ac51eb56e8c14b` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:45.173563 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1830d61b-f50d-49f4-b245-51fba3190ff0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=882f9799152040e394ac51eb56e8c14b)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:45.175455 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1830d61b-f50d-49f4-b245-51fba3190ff0 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:45.320975 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-1830d61b-f50d-49f4-b245-51fba3190ff0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '882f9799152040e394ac51eb56e8c14b'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:45.321912 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-1830d61b-f50d-49f4-b245-51fba3190ff0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '882f9799152040e394ac51eb56e8c14b'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:45.322938 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-1830d61b-f50d-49f4-b245-51fba3190ff0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '882f9799152040e394ac51eb56e8c14b'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:45.358165 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-1830d61b-f50d-49f4-b245-51fba3190ff0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '882f9799152040e394ac51eb56e8c14b'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:45.379518 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-1830d61b-f50d-49f4-b245-51fba3190ff0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '882f9799152040e394ac51eb56e8c14b'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:45.401516 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:45.411553 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 941/1874] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:26:44 2022] DELETE /identity/v3/users/882f9799152040e394ac51eb56e8c14b => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:45.717462 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db66e397-7915-4400-a8fd-9e68b43ba927 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:45.718997 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-db66e397-7915-4400-a8fd-9e68b43ba927 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:45.864378 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db66e397-7915-4400-a8fd-9e68b43ba927 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:45.968489 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c20ac-0d01-4959-9a56-35e947034b47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:45.969163 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c20ac-0d01-4959-9a56-35e947034b47 service cinder] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:45.969599 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c20ac-0d01-4959-9a56-35e947034b47 service cinder] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:45.969883 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c20ac-0d01-4959-9a56-35e947034b47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:45.972042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-357c20ac-0d01-4959-9a56-35e947034b47 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:46.015034 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:46.078263 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357c20ac-0d01-4959-9a56-35e947034b47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:46.079767 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357c20ac-0d01-4959-9a56-35e947034b47 service cinder] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:46.092230 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 934/1875] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:45 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:46.124305 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1830d61b-f50d-49f4-b245-51fba3190ff0 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:46.125851 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-1830d61b-f50d-49f4-b245-51fba3190ff0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:46.277831 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1830d61b-f50d-49f4-b245-51fba3190ff0 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:46.292125 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8422e3c5-1436-406a-a4d8-e2e0fcc02ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:46.293027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422e3c5-1436-406a-a4d8-e2e0fcc02ba0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:46.293498 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422e3c5-1436-406a-a4d8-e2e0fcc02ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:46.294014 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422e3c5-1436-406a-a4d8-e2e0fcc02ba0 admin admin] PATH_INFO: `/v3/projects/251046b70b0045148ac2cb70cd10581b` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:46.295515 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8422e3c5-1436-406a-a4d8-e2e0fcc02ba0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=251046b70b0045148ac2cb70cd10581b)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:46.297997 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8422e3c5-1436-406a-a4d8-e2e0fcc02ba0 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:46.382318 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-8422e3c5-1436-406a-a4d8-e2e0fcc02ba0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '251046b70b0045148ac2cb70cd10581b', 'request_id': 'req-8422e3c5-1436-406a-a4d8-e2e0fcc02ba0'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:46.389122 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-8422e3c5-1436-406a-a4d8-e2e0fcc02ba0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '251046b70b0045148ac2cb70cd10581b', 'request_id': 'req-8422e3c5-1436-406a-a4d8-e2e0fcc02ba0'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:46.394953 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-8422e3c5-1436-406a-a4d8-e2e0fcc02ba0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '251046b70b0045148ac2cb70cd10581b', 'request_id': 'req-8422e3c5-1436-406a-a4d8-e2e0fcc02ba0'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:46.417335 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-8422e3c5-1436-406a-a4d8-e2e0fcc02ba0 admin admin] The token cache is being invalidate because project 251046b70b0045148ac2cb70cd10581b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:26:46.417795 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-8422e3c5-1436-406a-a4d8-e2e0fcc02ba0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:46.418458 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-8422e3c5-1436-406a-a4d8-e2e0fcc02ba0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:46.427514 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 942/1876] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:26:46 2022] DELETE /identity/v3/projects/251046b70b0045148ac2cb70cd10581b => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:46.432252 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b598a0-8db9-4184-b0d3-cfb72c6faf96 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:46.432544 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b598a0-8db9-4184-b0d3-cfb72c6faf96 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:46.432846 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b598a0-8db9-4184-b0d3-cfb72c6faf96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:46.458903 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-83b598a0-8db9-4184-b0d3-cfb72c6faf96 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:46.460701 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-83b598a0-8db9-4184-b0d3-cfb72c6faf96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:46.532663 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 935/1877] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:46.541453 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3c310-c8b8-477a-b697-b144beba6f56 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:46.541807 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3c310-c8b8-477a-b697-b144beba6f56 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:46.542146 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3c310-c8b8-477a-b697-b144beba6f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:46.568112 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-baf3c310-c8b8-477a-b697-b144beba6f56 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:46.570429 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-baf3c310-c8b8-477a-b697-b144beba6f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:46.580108 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 943/1878] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:46.585343 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b598a0-8db9-4184-b0d3-cfb72c6faf96 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:46.595813 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b598a0-8db9-4184-b0d3-cfb72c6faf96 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:46.601719 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc675b00-1477-49a4-b4ec-3240080f5869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:46.602425 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc675b00-1477-49a4-b4ec-3240080f5869 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:46.602709 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc675b00-1477-49a4-b4ec-3240080f5869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:46.603011 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc675b00-1477-49a4-b4ec-3240080f5869 admin admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:46.604987 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc675b00-1477-49a4-b4ec-3240080f5869 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:26:46.604987 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc675b00-1477-49a4-b4ec-3240080f5869 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:46.610920 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc675b00-1477-49a4-b4ec-3240080f5869 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:46.623128 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 936/1879] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:26:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:26:46.627640 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf3c310-c8b8-477a-b697-b144beba6f56 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:46.639907 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf3c310-c8b8-477a-b697-b144beba6f56 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:46.645928 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120def66-1014-44b8-a06c-8061ce6fddaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:46.646800 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120def66-1014-44b8-a06c-8061ce6fddaa admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:46.647734 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120def66-1014-44b8-a06c-8061ce6fddaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:46.648204 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120def66-1014-44b8-a06c-8061ce6fddaa admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:46.648970 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120def66-1014-44b8-a06c-8061ce6fddaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:46.651174 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120def66-1014-44b8-a06c-8061ce6fddaa admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:46.679845 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 944/1880] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:26:46 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:26:46.683957 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc675b00-1477-49a4-b4ec-3240080f5869 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:46.694463 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc675b00-1477-49a4-b4ec-3240080f5869 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:46.758291 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138a7a49-7e9e-4dfb-8ad9-6a479b3a035e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:46.759534 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a7a49-7e9e-4dfb-8ad9-6a479b3a035e admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:46.760448 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a7a49-7e9e-4dfb-8ad9-6a479b3a035e admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:46.760448 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a7a49-7e9e-4dfb-8ad9-6a479b3a035e admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:46.760729 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138a7a49-7e9e-4dfb-8ad9-6a479b3a035e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:46.762684 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138a7a49-7e9e-4dfb-8ad9-6a479b3a035e admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:46.787027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 937/1881] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:26:46 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:26:46.791221 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120def66-1014-44b8-a06c-8061ce6fddaa admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:46.811097 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120def66-1014-44b8-a06c-8061ce6fddaa admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:46.823647 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73034fa4-ad49-44eb-a172-4c5489cb9d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:46.824853 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73034fa4-ad49-44eb-a172-4c5489cb9d65 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:46.825288 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73034fa4-ad49-44eb-a172-4c5489cb9d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:46.825832 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73034fa4-ad49-44eb-a172-4c5489cb9d65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:46.826547 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73034fa4-ad49-44eb-a172-4c5489cb9d65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:46.829021 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73034fa4-ad49-44eb-a172-4c5489cb9d65 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:46.845056 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 945/1882] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:46 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:46.848764 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138a7a49-7e9e-4dfb-8ad9-6a479b3a035e admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:46.864917 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138a7a49-7e9e-4dfb-8ad9-6a479b3a035e admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:46.875382 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5147c8aa-d18e-4f44-9471-10361f935ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:46.876520 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147c8aa-d18e-4f44-9471-10361f935ab9 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:46.876942 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147c8aa-d18e-4f44-9471-10361f935ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:46.877263 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147c8aa-d18e-4f44-9471-10361f935ab9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:46.877992 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5147c8aa-d18e-4f44-9471-10361f935ab9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:46.880002 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5147c8aa-d18e-4f44-9471-10361f935ab9 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:46.895199 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 938/1883] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:46 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:46.898649 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73034fa4-ad49-44eb-a172-4c5489cb9d65 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:46.928676 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73034fa4-ad49-44eb-a172-4c5489cb9d65 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:46.934041 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee26a8eb-c365-452a-bf98-81ca7515ed97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:46.934713 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26a8eb-c365-452a-bf98-81ca7515ed97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:46.935124 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26a8eb-c365-452a-bf98-81ca7515ed97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:46.935279 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26a8eb-c365-452a-bf98-81ca7515ed97 admin admin] PATH_INFO: `/v3/projects/9869121b0ac648388d9e94273fa70230/users/d304d6b3820946f3a784a6d43391de5c/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:46.955674 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee26a8eb-c365-452a-bf98-81ca7515ed97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9869121b0ac648388d9e94273fa70230, user_id=d304d6b3820946f3a784a6d43391de5c, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:46.958551 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee26a8eb-c365-452a-bf98-81ca7515ed97 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:46.972823 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 946/1884] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:46 2022] PUT /identity/v3/projects/9869121b0ac648388d9e94273fa70230/users/d304d6b3820946f3a784a6d43391de5c/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:46.982160 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd6668-429f-4411-be72-6f3cafd08ee5 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:46.982299 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd6668-429f-4411-be72-6f3cafd08ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:46.982509 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd6668-429f-4411-be72-6f3cafd08ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:46.989994 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee26a8eb-c365-452a-bf98-81ca7515ed97 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:46.992293 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-ee26a8eb-c365-452a-bf98-81ca7515ed97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:47.015232 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-2bbd6668-429f-4411-be72-6f3cafd08ee5 None None] MFA Rules not processed for user `d304d6b3820946f3a784a6d43391de5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:47.016583 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-2bbd6668-429f-4411-be72-6f3cafd08ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:47.025145 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:47.037398 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:47.070045 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee26a8eb-c365-452a-bf98-81ca7515ed97 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:47.075034 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9a6fa-3c94-46e9-a076-681000721fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:47.075669 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9a6fa-3c94-46e9-a076-681000721fa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:47.077345 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9a6fa-3c94-46e9-a076-681000721fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:47.077632 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9a6fa-3c94-46e9-a076-681000721fa7 admin admin] PATH_INFO: `/v3/users/0bbafb8836124f1a849e3168d4c5d194` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:47.078791 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd9a6fa-3c94-46e9-a076-681000721fa7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bbafb8836124f1a849e3168d4c5d194)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:47.080745 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd9a6fa-3c94-46e9-a076-681000721fa7 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:47.111223 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 939/1885] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:47.116415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbd6668-429f-4411-be72-6f3cafd08ee5 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:47.128617 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbd6668-429f-4411-be72-6f3cafd08ee5 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:47.131535 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9cd9a6fa-3c94-46e9-a076-681000721fa7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bbafb8836124f1a849e3168d4c5d194'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:47.132202 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9cd9a6fa-3c94-46e9-a076-681000721fa7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bbafb8836124f1a849e3168d4c5d194'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:47.132907 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9cd9a6fa-3c94-46e9-a076-681000721fa7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bbafb8836124f1a849e3168d4c5d194'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:47.134022 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce42bda-676d-4d18-a4f8-8def4eea4a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:47.134729 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce42bda-676d-4d18-a4f8-8def4eea4a7b admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:47.135130 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce42bda-676d-4d18-a4f8-8def4eea4a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:47.135473 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce42bda-676d-4d18-a4f8-8def4eea4a7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:47.136212 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce42bda-676d-4d18-a4f8-8def4eea4a7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:47.138246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce42bda-676d-4d18-a4f8-8def4eea4a7b admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:47.150580 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9cd9a6fa-3c94-46e9-a076-681000721fa7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bbafb8836124f1a849e3168d4c5d194'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:47.163280 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9cd9a6fa-3c94-46e9-a076-681000721fa7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bbafb8836124f1a849e3168d4c5d194'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:47.169557 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 940/1886] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:26:47 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:47.173706 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce42bda-676d-4d18-a4f8-8def4eea4a7b admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:47.175345 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-cce42bda-676d-4d18-a4f8-8def4eea4a7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:47.178163 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 947/1887] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:26:46 2022] DELETE /identity/v3/users/0bbafb8836124f1a849e3168d4c5d194 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:26:47.182904 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a087f0-3104-46b9-bed5-2a669c467d59 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:47.183180 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a087f0-3104-46b9-bed5-2a669c467d59 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:47.183392 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a087f0-3104-46b9-bed5-2a669c467d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:47.204397 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-f2a087f0-3104-46b9-bed5-2a669c467d59 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:47.205678 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-f2a087f0-3104-46b9-bed5-2a669c467d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:47.246880 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce42bda-676d-4d18-a4f8-8def4eea4a7b admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:47.268706 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 948/1888] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:47.279083 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a087f0-3104-46b9-bed5-2a669c467d59 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:47.280753 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-f2a087f0-3104-46b9-bed5-2a669c467d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:47.309270 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407adabc-88de-48df-8f32-a3b50a80568a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:47.309885 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407adabc-88de-48df-8f32-a3b50a80568a admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:47.310163 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407adabc-88de-48df-8f32-a3b50a80568a admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:47.310435 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407adabc-88de-48df-8f32-a3b50a80568a admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:47.311472 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407adabc-88de-48df-8f32-a3b50a80568a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:47.313053 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407adabc-88de-48df-8f32-a3b50a80568a admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:47.339545 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 941/1889] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:26:47 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:47.339745 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:47.340878 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407adabc-88de-48df-8f32-a3b50a80568a admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:47.342583 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a087f0-3104-46b9-bed5-2a669c467d59 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:47.347813 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a72c4-2d92-45c6-ba63-c39e46fc1d16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:47.348625 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a72c4-2d92-45c6-ba63-c39e46fc1d16 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:47.348871 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a72c4-2d92-45c6-ba63-c39e46fc1d16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:47.349300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a72c4-2d92-45c6-ba63-c39e46fc1d16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:47.351432 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407adabc-88de-48df-8f32-a3b50a80568a admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:47.356136 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b24902-00da-4bd2-89af-11bc794e2419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:47.356725 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b24902-00da-4bd2-89af-11bc794e2419 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:47.357005 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b24902-00da-4bd2-89af-11bc794e2419 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:47.357276 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b24902-00da-4bd2-89af-11bc794e2419 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:47.357911 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b24902-00da-4bd2-89af-11bc794e2419 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:47.358942 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4a72c4-2d92-45c6-ba63-c39e46fc1d16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:47.359573 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b24902-00da-4bd2-89af-11bc794e2419 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:47.360366 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4a72c4-2d92-45c6-ba63-c39e46fc1d16 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:47.373153 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 942/1890] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:47.373291 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 949/1891] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:47.375596 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b24902-00da-4bd2-89af-11bc794e2419 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:47.385948 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b24902-00da-4bd2-89af-11bc794e2419 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:47.391742 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b859420-261f-43a3-a6a0-a66c96dfc5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:47.392439 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b859420-261f-43a3-a6a0-a66c96dfc5ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:47.392721 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b859420-261f-43a3-a6a0-a66c96dfc5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:47.392995 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b859420-261f-43a3-a6a0-a66c96dfc5ac admin admin] PATH_INFO: `/v3/projects/80101e88df744193b7685bcea9b4f97b/users/3d4ffe7637984a9482a81016a8401d93/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:47.415233 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b859420-261f-43a3-a6a0-a66c96dfc5ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=80101e88df744193b7685bcea9b4f97b, user_id=3d4ffe7637984a9482a81016a8401d93, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:47.417896 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b859420-261f-43a3-a6a0-a66c96dfc5ac admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:47.425922 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:47.432641 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 943/1892] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:47 2022] PUT /identity/v3/projects/80101e88df744193b7685bcea9b4f97b/users/3d4ffe7637984a9482a81016a8401d93/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:26:47.436557 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a72c4-2d92-45c6-ba63-c39e46fc1d16 service neutron] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:47.446881 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a72c4-2d92-45c6-ba63-c39e46fc1d16 service neutron] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:47.504444 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d5c206-be83-48ca-af9c-1ca450b9c3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:47.505088 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5c206-be83-48ca-af9c-1ca450b9c3e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:47.505480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5c206-be83-48ca-af9c-1ca450b9c3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:47.505752 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5c206-be83-48ca-af9c-1ca450b9c3e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:47.506990 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d5c206-be83-48ca-af9c-1ca450b9c3e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:47.508591 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d5c206-be83-48ca-af9c-1ca450b9c3e9 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:47.520280 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 950/1893] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:47 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:47.525226 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b859420-261f-43a3-a6a0-a66c96dfc5ac admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:47.537096 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b859420-261f-43a3-a6a0-a66c96dfc5ac admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:47.541800 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad37aaa-64c3-41b1-8e11-4c2c5f7927b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:47.542540 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad37aaa-64c3-41b1-8e11-4c2c5f7927b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:47.542911 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad37aaa-64c3-41b1-8e11-4c2c5f7927b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:47.543360 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad37aaa-64c3-41b1-8e11-4c2c5f7927b3 admin admin] PATH_INFO: `/v3/projects/80101e88df744193b7685bcea9b4f97b/users/3d4ffe7637984a9482a81016a8401d93/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:47.545216 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad37aaa-64c3-41b1-8e11-4c2c5f7927b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80101e88df744193b7685bcea9b4f97b, user_id=3d4ffe7637984a9482a81016a8401d93, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:47.547957 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad37aaa-64c3-41b1-8e11-4c2c5f7927b3 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:47.558739 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 944/1894] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:47 2022] PUT /identity/v3/projects/80101e88df744193b7685bcea9b4f97b/users/3d4ffe7637984a9482a81016a8401d93/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:47.570717 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875f9a0-3bc1-473d-bfda-8b9739228a08 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:47.571029 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875f9a0-3bc1-473d-bfda-8b9739228a08 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:47.571303 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875f9a0-3bc1-473d-bfda-8b9739228a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:47.602271 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-4875f9a0-3bc1-473d-bfda-8b9739228a08 None None] MFA Rules not processed for user `3d4ffe7637984a9482a81016a8401d93`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:47.603543 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-4875f9a0-3bc1-473d-bfda-8b9739228a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:47.652459 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad37aaa-64c3-41b1-8e11-4c2c5f7927b3 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:47.654675 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-aad37aaa-64c3-41b1-8e11-4c2c5f7927b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:47.696690 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 951/1895] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:47 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:26:47.709107 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4875f9a0-3bc1-473d-bfda-8b9739228a08 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:47.710464 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-4875f9a0-3bc1-473d-bfda-8b9739228a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:47.751872 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad37aaa-64c3-41b1-8e11-4c2c5f7927b3 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:47.757028 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473fe4b3-7a4e-495f-9295-89078b1a7ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:47.757649 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473fe4b3-7a4e-495f-9295-89078b1a7ca8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:47.757928 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473fe4b3-7a4e-495f-9295-89078b1a7ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:47.758196 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473fe4b3-7a4e-495f-9295-89078b1a7ca8 admin admin] PATH_INFO: `/v3/projects/986612297af94de284366296a6fd269f` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:47.759416 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473fe4b3-7a4e-495f-9295-89078b1a7ca8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=986612297af94de284366296a6fd269f)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:47.761354 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473fe4b3-7a4e-495f-9295-89078b1a7ca8 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:47.770506 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4875f9a0-3bc1-473d-bfda-8b9739228a08 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:47.774751 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfe785e-128a-4766-acc8-b3d488777a0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:47.775393 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe785e-128a-4766-acc8-b3d488777a0e service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:47.775673 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe785e-128a-4766-acc8-b3d488777a0e service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:47.775949 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe785e-128a-4766-acc8-b3d488777a0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:47.785411 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfe785e-128a-4766-acc8-b3d488777a0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:47.786868 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfe785e-128a-4766-acc8-b3d488777a0e service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:47.797290 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 952/1896] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:47 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 14:26:47.811074 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-473fe4b3-7a4e-495f-9295-89078b1a7ca8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '986612297af94de284366296a6fd269f', 'request_id': 'req-473fe4b3-7a4e-495f-9295-89078b1a7ca8'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:47.817783 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-473fe4b3-7a4e-495f-9295-89078b1a7ca8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '986612297af94de284366296a6fd269f', 'request_id': 'req-473fe4b3-7a4e-495f-9295-89078b1a7ca8'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:47.825574 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-473fe4b3-7a4e-495f-9295-89078b1a7ca8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '986612297af94de284366296a6fd269f', 'request_id': 'req-473fe4b3-7a4e-495f-9295-89078b1a7ca8'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:47.843011 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-473fe4b3-7a4e-495f-9295-89078b1a7ca8 admin admin] The token cache is being invalidate because project 986612297af94de284366296a6fd269f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:26:47.843305 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-473fe4b3-7a4e-495f-9295-89078b1a7ca8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:47.843950 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-473fe4b3-7a4e-495f-9295-89078b1a7ca8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:47.845482 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 945/1897] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:26:47 2022] DELETE /identity/v3/projects/986612297af94de284366296a6fd269f => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:47.849298 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfe785e-128a-4766-acc8-b3d488777a0e service cinder] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:47.859650 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfe785e-128a-4766-acc8-b3d488777a0e service cinder] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:47.871794 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:47.926435 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf589630-4a9e-41c4-a4a9-7d6bc7d16c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:47.927103 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf589630-4a9e-41c4-a4a9-7d6bc7d16c2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:47.927380 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf589630-4a9e-41c4-a4a9-7d6bc7d16c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:47.927653 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf589630-4a9e-41c4-a4a9-7d6bc7d16c2b admin admin] PATH_INFO: `/v3/users/c1c8120fb76c4dca9bbf78cfee16a76b` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:47.929290 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf589630-4a9e-41c4-a4a9-7d6bc7d16c2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1c8120fb76c4dca9bbf78cfee16a76b)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:47.931230 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf589630-4a9e-41c4-a4a9-7d6bc7d16c2b admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:47.986623 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-bf589630-4a9e-41c4-a4a9-7d6bc7d16c2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1c8120fb76c4dca9bbf78cfee16a76b'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:47.987372 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-bf589630-4a9e-41c4-a4a9-7d6bc7d16c2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1c8120fb76c4dca9bbf78cfee16a76b'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:47.987992 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-bf589630-4a9e-41c4-a4a9-7d6bc7d16c2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1c8120fb76c4dca9bbf78cfee16a76b'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:48.002166 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-bf589630-4a9e-41c4-a4a9-7d6bc7d16c2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1c8120fb76c4dca9bbf78cfee16a76b'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:48.012010 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-bf589630-4a9e-41c4-a4a9-7d6bc7d16c2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1c8120fb76c4dca9bbf78cfee16a76b'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:48.025659 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 953/1898] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:26:47 2022] DELETE /identity/v3/users/c1c8120fb76c4dca9bbf78cfee16a76b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:48.155418 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473fe4b3-7a4e-495f-9295-89078b1a7ca8 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:48.157557 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-473fe4b3-7a4e-495f-9295-89078b1a7ca8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:48.247765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473fe4b3-7a4e-495f-9295-89078b1a7ca8 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:48.279174 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:48.330988 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60c78aa-8037-4daa-a968-0dfa37a702b0 tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d4ffe7637984a9482a81016a8401d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80101e88df744193b7685bcea9b4f97b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:48.331908 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c78aa-8037-4daa-a968-0dfa37a702b0 tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:48.332376 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c78aa-8037-4daa-a968-0dfa37a702b0 tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:48.332840 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c78aa-8037-4daa-a968-0dfa37a702b0 tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:48.333638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60c78aa-8037-4daa-a968-0dfa37a702b0 tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:26:48.334572 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60c78aa-8037-4daa-a968-0dfa37a702b0 tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:48.336871 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60c78aa-8037-4daa-a968-0dfa37a702b0 tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:48.351248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 946/1899] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:26:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:26:48.355271 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf589630-4a9e-41c4-a4a9-7d6bc7d16c2b admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:48.356860 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-bf589630-4a9e-41c4-a4a9-7d6bc7d16c2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:48.371251 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf589630-4a9e-41c4-a4a9-7d6bc7d16c2b admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:48.375881 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42537870-637f-49e9-a537-25bec6ef199b tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d4ffe7637984a9482a81016a8401d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80101e88df744193b7685bcea9b4f97b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:48.376509 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42537870-637f-49e9-a537-25bec6ef199b tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:48.376792 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42537870-637f-49e9-a537-25bec6ef199b tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:48.377068 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42537870-637f-49e9-a537-25bec6ef199b tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:48.377727 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42537870-637f-49e9-a537-25bec6ef199b tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:48.379691 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42537870-637f-49e9-a537-25bec6ef199b tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:48.413160 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 954/1900] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:26:48 2022] POST /identity/v3/projects => generated 354 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:48.414984 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60c78aa-8037-4daa-a968-0dfa37a702b0 tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:48.426405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60c78aa-8037-4daa-a968-0dfa37a702b0 tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:48.493990 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9db080-2c7f-46e1-9ada-2ee5e13f2cb1 tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d4ffe7637984a9482a81016a8401d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80101e88df744193b7685bcea9b4f97b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:48.494410 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9db080-2c7f-46e1-9ada-2ee5e13f2cb1 tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:48.494639 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9db080-2c7f-46e1-9ada-2ee5e13f2cb1 tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:48.494892 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9db080-2c7f-46e1-9ada-2ee5e13f2cb1 tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:48.495911 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9db080-2c7f-46e1-9ada-2ee5e13f2cb1 tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:26:48.495911 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9db080-2c7f-46e1-9ada-2ee5e13f2cb1 tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:48.497558 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9db080-2c7f-46e1-9ada-2ee5e13f2cb1 tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:48.515405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 947/1901] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:26:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:26:48.547834 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42537870-637f-49e9-a537-25bec6ef199b tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:48.550972 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-42537870-637f-49e9-a537-25bec6ef199b tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:48.560335 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9db080-2c7f-46e1-9ada-2ee5e13f2cb1 tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:48.571616 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9db080-2c7f-46e1-9ada-2ee5e13f2cb1 tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:48.576410 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf5fb12-3db6-427f-b1e1-5a9ecec23a96 tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d4ffe7637984a9482a81016a8401d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80101e88df744193b7685bcea9b4f97b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:48.577065 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf5fb12-3db6-427f-b1e1-5a9ecec23a96 tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:48.577331 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf5fb12-3db6-427f-b1e1-5a9ecec23a96 tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:48.577636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf5fb12-3db6-427f-b1e1-5a9ecec23a96 tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] PATH_INFO: `/v3/projects/38d50dddddde4c10bd7c4318d92ff848` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:48.578767 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf5fb12-3db6-427f-b1e1-5a9ecec23a96 tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] RBAC: Authorizing `identity:delete_project(project_id=38d50dddddde4c10bd7c4318d92ff848)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:48.580944 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf5fb12-3db6-427f-b1e1-5a9ecec23a96 tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:48.642864 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42537870-637f-49e9-a537-25bec6ef199b tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:48.652300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:48.665668 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-dcf5fb12-3db6-427f-b1e1-5a9ecec23a96 tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38d50dddddde4c10bd7c4318d92ff848', 'request_id': 'req-dcf5fb12-3db6-427f-b1e1-5a9ecec23a96'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:48.672841 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-dcf5fb12-3db6-427f-b1e1-5a9ecec23a96 tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38d50dddddde4c10bd7c4318d92ff848', 'request_id': 'req-dcf5fb12-3db6-427f-b1e1-5a9ecec23a96'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:48.683023 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-dcf5fb12-3db6-427f-b1e1-5a9ecec23a96 tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38d50dddddde4c10bd7c4318d92ff848', 'request_id': 'req-dcf5fb12-3db6-427f-b1e1-5a9ecec23a96'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:48.697213 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-dcf5fb12-3db6-427f-b1e1-5a9ecec23a96 tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] The token cache is being invalidate because project 38d50dddddde4c10bd7c4318d92ff848 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:26:48.697508 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-dcf5fb12-3db6-427f-b1e1-5a9ecec23a96 tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:48.698616 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-dcf5fb12-3db6-427f-b1e1-5a9ecec23a96 tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:48.700311 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 948/1902] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:26:48 2022] DELETE /identity/v3/projects/38d50dddddde4c10bd7c4318d92ff848 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:26:48.701092 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf5fb12-3db6-427f-b1e1-5a9ecec23a96 tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:48.702510 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-dcf5fb12-3db6-427f-b1e1-5a9ecec23a96 tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:48.714424 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb76cad4-49e0-432f-a919-71b58f078ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:48.715140 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76cad4-49e0-432f-a919-71b58f078ae5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:48.715494 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76cad4-49e0-432f-a919-71b58f078ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:48.715774 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76cad4-49e0-432f-a919-71b58f078ae5 admin admin] PATH_INFO: `/v3/users/513f7faab9a041a999478133549a2590` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:48.716930 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb76cad4-49e0-432f-a919-71b58f078ae5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=513f7faab9a041a999478133549a2590)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:48.718901 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb76cad4-49e0-432f-a919-71b58f078ae5 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:48.749453 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:48.779691 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-fb76cad4-49e0-432f-a919-71b58f078ae5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '513f7faab9a041a999478133549a2590'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:48.780478 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-fb76cad4-49e0-432f-a919-71b58f078ae5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '513f7faab9a041a999478133549a2590'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:48.781501 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-fb76cad4-49e0-432f-a919-71b58f078ae5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '513f7faab9a041a999478133549a2590'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:48.786400 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf5fb12-3db6-427f-b1e1-5a9ecec23a96 tempest-VolumeQuotaClassesTest-1815968510 tempest-VolumeQuotaClassesTest-1815968510-project-admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:48.792067 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06044416-6570-48f7-b9bd-8f3b4aac799e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:48.792785 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06044416-6570-48f7-b9bd-8f3b4aac799e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:48.793130 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06044416-6570-48f7-b9bd-8f3b4aac799e admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:48.793479 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06044416-6570-48f7-b9bd-8f3b4aac799e admin admin] PATH_INFO: `/v3/projects/6db5619ed1c1414985e51d4faed65e75` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:48.794677 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06044416-6570-48f7-b9bd-8f3b4aac799e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6db5619ed1c1414985e51d4faed65e75)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:48.796706 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06044416-6570-48f7-b9bd-8f3b4aac799e admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:48.796860 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-fb76cad4-49e0-432f-a919-71b58f078ae5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '513f7faab9a041a999478133549a2590'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:48.805680 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-fb76cad4-49e0-432f-a919-71b58f078ae5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '513f7faab9a041a999478133549a2590'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:48.822216 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 955/1903] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:26:48 2022] DELETE /identity/v3/users/513f7faab9a041a999478133549a2590 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:48.827538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9266c895-9a93-4091-8e61-3416cb3f2812 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:48.827911 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9266c895-9a93-4091-8e61-3416cb3f2812 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:48.828217 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9266c895-9a93-4091-8e61-3416cb3f2812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:48.849380 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-9266c895-9a93-4091-8e61-3416cb3f2812 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:48.851247 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-9266c895-9a93-4091-8e61-3416cb3f2812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:48.864029 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-06044416-6570-48f7-b9bd-8f3b4aac799e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6db5619ed1c1414985e51d4faed65e75', 'request_id': 'req-06044416-6570-48f7-b9bd-8f3b4aac799e'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:48.873550 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-06044416-6570-48f7-b9bd-8f3b4aac799e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6db5619ed1c1414985e51d4faed65e75', 'request_id': 'req-06044416-6570-48f7-b9bd-8f3b4aac799e'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:48.880253 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-06044416-6570-48f7-b9bd-8f3b4aac799e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6db5619ed1c1414985e51d4faed65e75', 'request_id': 'req-06044416-6570-48f7-b9bd-8f3b4aac799e'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:48.894741 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-06044416-6570-48f7-b9bd-8f3b4aac799e admin admin] The token cache is being invalidate because project 6db5619ed1c1414985e51d4faed65e75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:26:48.895356 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-06044416-6570-48f7-b9bd-8f3b4aac799e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:48.896276 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-06044416-6570-48f7-b9bd-8f3b4aac799e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:48.898240 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 949/1904] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:26:48 2022] DELETE /identity/v3/projects/6db5619ed1c1414985e51d4faed65e75 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:48.903125 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c92c66-11af-4467-b876-68446c0a41e2 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:48.903419 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c92c66-11af-4467-b876-68446c0a41e2 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:48.903751 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c92c66-11af-4467-b876-68446c0a41e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:48.920627 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 956/1905] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:48.927462 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-25c92c66-11af-4467-b876-68446c0a41e2 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:48.929316 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-25c92c66-11af-4467-b876-68446c0a41e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:48.935018 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9266c895-9a93-4091-8e61-3416cb3f2812 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:48.936481 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-9266c895-9a93-4091-8e61-3416cb3f2812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:48.937648 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 950/1906] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:48.938554 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c92c66-11af-4467-b876-68446c0a41e2 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:48.939808 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-25c92c66-11af-4467-b876-68446c0a41e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:48.952437 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c92c66-11af-4467-b876-68446c0a41e2 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:48.956965 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b086a5a-ca45-4b07-bc07-629c71eb92ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:48.957579 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b086a5a-ca45-4b07-bc07-629c71eb92ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:48.957855 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b086a5a-ca45-4b07-bc07-629c71eb92ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:48.958120 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b086a5a-ca45-4b07-bc07-629c71eb92ba admin admin] PATH_INFO: `/v3/users/d304d6b3820946f3a784a6d43391de5c` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:48.959289 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b086a5a-ca45-4b07-bc07-629c71eb92ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=d304d6b3820946f3a784a6d43391de5c)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:48.961192 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b086a5a-ca45-4b07-bc07-629c71eb92ba admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:48.984394 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:49.002503 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9266c895-9a93-4091-8e61-3416cb3f2812 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:49.006199 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d446ba-1359-452c-98dc-09d1c5dfa0c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:49.006878 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d446ba-1359-452c-98dc-09d1c5dfa0c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:49.007161 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d446ba-1359-452c-98dc-09d1c5dfa0c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:49.007433 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d446ba-1359-452c-98dc-09d1c5dfa0c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:49.010272 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-7b086a5a-ca45-4b07-bc07-629c71eb92ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd304d6b3820946f3a784a6d43391de5c'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:49.010967 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-7b086a5a-ca45-4b07-bc07-629c71eb92ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd304d6b3820946f3a784a6d43391de5c'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:49.017317 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d446ba-1359-452c-98dc-09d1c5dfa0c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:49.017996 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-7b086a5a-ca45-4b07-bc07-629c71eb92ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd304d6b3820946f3a784a6d43391de5c'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:49.018730 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-7b086a5a-ca45-4b07-bc07-629c71eb92ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd304d6b3820946f3a784a6d43391de5c'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:49.018846 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d446ba-1359-452c-98dc-09d1c5dfa0c9 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:49.028651 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 957/1907] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 14:26:49.029951 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3705e9b-132f-422d-9edf-c3e99ab27ecb None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:49.030239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3705e9b-132f-422d-9edf-c3e99ab27ecb None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:49.030512 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3705e9b-132f-422d-9edf-c3e99ab27ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:49.031729 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-7b086a5a-ca45-4b07-bc07-629c71eb92ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd304d6b3820946f3a784a6d43391de5c'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:49.047398 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 951/1908] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:26:48 2022] DELETE /identity/v3/users/d304d6b3820946f3a784a6d43391de5c => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:49.052368 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff546d59-56ad-40bd-9248-2f69c24e902f None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:49.052656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff546d59-56ad-40bd-9248-2f69c24e902f None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:49.053553 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff546d59-56ad-40bd-9248-2f69c24e902f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:49.055671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-e3705e9b-132f-422d-9edf-c3e99ab27ecb None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:49.057148 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-e3705e9b-132f-422d-9edf-c3e99ab27ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:49.075264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-ff546d59-56ad-40bd-9248-2f69c24e902f None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:49.076532 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-ff546d59-56ad-40bd-9248-2f69c24e902f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:49.107597 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:49.138863 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 958/1909] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:49.141190 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3705e9b-132f-422d-9edf-c3e99ab27ecb None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:49.154333 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3705e9b-132f-422d-9edf-c3e99ab27ecb None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:49.157579 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 952/1910] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:49.160356 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a60f4-cb26-4931-986a-22aabbf5c758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:49.160973 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a60f4-cb26-4931-986a-22aabbf5c758 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:49.161254 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a60f4-cb26-4931-986a-22aabbf5c758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:49.161636 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a60f4-cb26-4931-986a-22aabbf5c758 admin admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:49.162838 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922a60f4-cb26-4931-986a-22aabbf5c758 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:26:49.163147 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922a60f4-cb26-4931-986a-22aabbf5c758 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:49.164769 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922a60f4-cb26-4931-986a-22aabbf5c758 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:49.169317 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff546d59-56ad-40bd-9248-2f69c24e902f None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:49.170991 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-ff546d59-56ad-40bd-9248-2f69c24e902f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:49.177630 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 959/1911] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:26:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:26:49.194436 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a60f4-cb26-4931-986a-22aabbf5c758 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:49.213080 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a60f4-cb26-4931-986a-22aabbf5c758 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:49.221347 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0621d9ce-f7a3-4396-8ed5-0a2a9a6ad6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:49.222581 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621d9ce-f7a3-4396-8ed5-0a2a9a6ad6f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:49.223708 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621d9ce-f7a3-4396-8ed5-0a2a9a6ad6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:49.224171 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621d9ce-f7a3-4396-8ed5-0a2a9a6ad6f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:49.225589 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0621d9ce-f7a3-4396-8ed5-0a2a9a6ad6f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:49.228310 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0621d9ce-f7a3-4396-8ed5-0a2a9a6ad6f7 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:49.245564 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff546d59-56ad-40bd-9248-2f69c24e902f None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:49.249849 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e077c5d-67e4-419c-90b3-82efa292be16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:49.250552 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e077c5d-67e4-419c-90b3-82efa292be16 service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:49.250877 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e077c5d-67e4-419c-90b3-82efa292be16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:49.251127 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e077c5d-67e4-419c-90b3-82efa292be16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:49.261583 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 960/1912] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:26:49 2022] POST /identity/v3/projects => generated 381 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:49.265238 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e077c5d-67e4-419c-90b3-82efa292be16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:49.265461 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e077c5d-67e4-419c-90b3-82efa292be16 service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:49.267890 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0621d9ce-f7a3-4396-8ed5-0a2a9a6ad6f7 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:49.281416 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 953/1913] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:49.336023 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0621d9ce-f7a3-4396-8ed5-0a2a9a6ad6f7 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:49.341985 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13cb8f8-58f8-4af5-b23b-c34ed258cd9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:49.342595 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13cb8f8-58f8-4af5-b23b-c34ed258cd9c admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:49.342897 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13cb8f8-58f8-4af5-b23b-c34ed258cd9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:49.343170 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13cb8f8-58f8-4af5-b23b-c34ed258cd9c admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:49.343809 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13cb8f8-58f8-4af5-b23b-c34ed258cd9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:49.345766 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13cb8f8-58f8-4af5-b23b-c34ed258cd9c admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:49.382878 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 961/1914] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:26:49 2022] POST /identity/v3/users => generated 291 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:49.388709 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e077c5d-67e4-419c-90b3-82efa292be16 service neutron] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:49.400244 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e077c5d-67e4-419c-90b3-82efa292be16 service neutron] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:49.405964 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86afd3af-4f39-4109-99e8-6eabb3e2b540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:49.406751 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86afd3af-4f39-4109-99e8-6eabb3e2b540 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:49.407067 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86afd3af-4f39-4109-99e8-6eabb3e2b540 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:49.407336 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86afd3af-4f39-4109-99e8-6eabb3e2b540 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:49.407976 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86afd3af-4f39-4109-99e8-6eabb3e2b540 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:49.409818 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86afd3af-4f39-4109-99e8-6eabb3e2b540 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:49.422112 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 954/1915] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:49.427284 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13cb8f8-58f8-4af5-b23b-c34ed258cd9c admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:49.438702 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13cb8f8-58f8-4af5-b23b-c34ed258cd9c admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:49.445955 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86afd3af-4f39-4109-99e8-6eabb3e2b540 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:49.447632 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a32490-33c4-44b5-98f2-0bc359837bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:49.448235 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-86afd3af-4f39-4109-99e8-6eabb3e2b540 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:49.448359 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a32490-33c4-44b5-98f2-0bc359837bfc admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:49.448553 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a32490-33c4-44b5-98f2-0bc359837bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:49.448774 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a32490-33c4-44b5-98f2-0bc359837bfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:49.449525 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a32490-33c4-44b5-98f2-0bc359837bfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:49.451030 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a32490-33c4-44b5-98f2-0bc359837bfc admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:49.480668 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 962/1916] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:49.481826 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a32490-33c4-44b5-98f2-0bc359837bfc admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:49.484251 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86afd3af-4f39-4109-99e8-6eabb3e2b540 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:49.498121 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe3952e-d93b-4a6f-917f-86c3ea9c8692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:49.498771 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe3952e-d93b-4a6f-917f-86c3ea9c8692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:49.499136 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe3952e-d93b-4a6f-917f-86c3ea9c8692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:49.499413 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe3952e-d93b-4a6f-917f-86c3ea9c8692 admin admin] PATH_INFO: `/v3/projects/652ebf19122e47449d07e6c37a2f7578` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:49.500723 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe3952e-d93b-4a6f-917f-86c3ea9c8692 admin admin] RBAC: Authorizing `identity:delete_project(project_id=652ebf19122e47449d07e6c37a2f7578)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:49.502360 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a32490-33c4-44b5-98f2-0bc359837bfc admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:49.502705 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe3952e-d93b-4a6f-917f-86c3ea9c8692 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:49.517047 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17000651-c0eb-480f-8601-63705e837e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:49.517983 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17000651-c0eb-480f-8601-63705e837e1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:49.518183 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17000651-c0eb-480f-8601-63705e837e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:49.518568 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17000651-c0eb-480f-8601-63705e837e1f admin admin] PATH_INFO: `/v3/projects/9a33b93133414161a03bde9c002a7e3e/users/33084eda6b6045d3a031f452d1a05b95/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:49.554896 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17000651-c0eb-480f-8601-63705e837e1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a33b93133414161a03bde9c002a7e3e, user_id=33084eda6b6045d3a031f452d1a05b95, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:49.557603 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17000651-c0eb-480f-8601-63705e837e1f admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:49.569162 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:49.584728 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-abe3952e-d93b-4a6f-917f-86c3ea9c8692 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '652ebf19122e47449d07e6c37a2f7578', 'request_id': 'req-abe3952e-d93b-4a6f-917f-86c3ea9c8692'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:49.594744 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-abe3952e-d93b-4a6f-917f-86c3ea9c8692 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '652ebf19122e47449d07e6c37a2f7578', 'request_id': 'req-abe3952e-d93b-4a6f-917f-86c3ea9c8692'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:49.600989 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 963/1917] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:49 2022] PUT /identity/v3/projects/9a33b93133414161a03bde9c002a7e3e/users/33084eda6b6045d3a031f452d1a05b95/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:49.603169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-abe3952e-d93b-4a6f-917f-86c3ea9c8692 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '652ebf19122e47449d07e6c37a2f7578', 'request_id': 'req-abe3952e-d93b-4a6f-917f-86c3ea9c8692'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:49.603319 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00947eb3-a166-4591-b935-02515030a7d0 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:49.603401 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00947eb3-a166-4591-b935-02515030a7d0 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:49.603953 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00947eb3-a166-4591-b935-02515030a7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:49.619288 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-abe3952e-d93b-4a6f-917f-86c3ea9c8692 admin admin] The token cache is being invalidate because project 652ebf19122e47449d07e6c37a2f7578 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:26:49.619829 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-abe3952e-d93b-4a6f-917f-86c3ea9c8692 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:49.620493 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-abe3952e-d93b-4a6f-917f-86c3ea9c8692 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:49.625013 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 955/1918] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:26:49 2022] DELETE /identity/v3/projects/652ebf19122e47449d07e6c37a2f7578 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:26:49.626283 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe3952e-d93b-4a6f-917f-86c3ea9c8692 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:49.627888 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-abe3952e-d93b-4a6f-917f-86c3ea9c8692 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:49.637263 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-00947eb3-a166-4591-b935-02515030a7d0 None None] MFA Rules not processed for user `33084eda6b6045d3a031f452d1a05b95`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:49.638600 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-00947eb3-a166-4591-b935-02515030a7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:49.695199 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:49.728186 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 964/1919] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:49.732876 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00947eb3-a166-4591-b935-02515030a7d0 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:49.734847 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-00947eb3-a166-4591-b935-02515030a7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:49.761695 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe3952e-d93b-4a6f-917f-86c3ea9c8692 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:49.826719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00947eb3-a166-4591-b935-02515030a7d0 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:49.837334 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ae314-1d54-4eda-b1a3-2e8a6fe5d5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:49.838037 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ae314-1d54-4eda-b1a3-2e8a6fe5d5e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:49.838256 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ae314-1d54-4eda-b1a3-2e8a6fe5d5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:49.838508 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ae314-1d54-4eda-b1a3-2e8a6fe5d5e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:49.839280 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008ae314-1d54-4eda-b1a3-2e8a6fe5d5e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:49.841396 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008ae314-1d54-4eda-b1a3-2e8a6fe5d5e5 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:49.854114 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ef609-f09b-4ae9-8a9f-ee505caf2f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:49.854750 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ef609-f09b-4ae9-8a9f-ee505caf2f0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:49.855094 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ef609-f09b-4ae9-8a9f-ee505caf2f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:49.855374 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ef609-f09b-4ae9-8a9f-ee505caf2f0c admin admin] PATH_INFO: `/v3/users/33558a40a4984a00bea603e5ae8d68d6` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:49.856813 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078ef609-f09b-4ae9-8a9f-ee505caf2f0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=33558a40a4984a00bea603e5ae8d68d6)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:49.858739 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078ef609-f09b-4ae9-8a9f-ee505caf2f0c admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:49.884329 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 965/1920] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:26:49 2022] POST /identity/v3/projects => generated 381 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:26:49.884329 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ae314-1d54-4eda-b1a3-2e8a6fe5d5e5 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:49.884329 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-008ae314-1d54-4eda-b1a3-2e8a6fe5d5e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:49.907979 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-078ef609-f09b-4ae9-8a9f-ee505caf2f0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33558a40a4984a00bea603e5ae8d68d6'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:49.909352 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-078ef609-f09b-4ae9-8a9f-ee505caf2f0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33558a40a4984a00bea603e5ae8d68d6'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:49.917212 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-078ef609-f09b-4ae9-8a9f-ee505caf2f0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33558a40a4984a00bea603e5ae8d68d6'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:49.918027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-078ef609-f09b-4ae9-8a9f-ee505caf2f0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33558a40a4984a00bea603e5ae8d68d6'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:49.934016 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-078ef609-f09b-4ae9-8a9f-ee505caf2f0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33558a40a4984a00bea603e5ae8d68d6'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:49.954882 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ae314-1d54-4eda-b1a3-2e8a6fe5d5e5 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:49.960164 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e52f201-2089-40b9-92a6-96b22f833721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:49.960889 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e52f201-2089-40b9-92a6-96b22f833721 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:49.961185 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e52f201-2089-40b9-92a6-96b22f833721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:49.961654 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e52f201-2089-40b9-92a6-96b22f833721 admin admin] PATH_INFO: `/v3/projects/9869121b0ac648388d9e94273fa70230` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:49.962824 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e52f201-2089-40b9-92a6-96b22f833721 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9869121b0ac648388d9e94273fa70230)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:49.965117 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e52f201-2089-40b9-92a6-96b22f833721 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:49.971904 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 956/1921] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:26:49 2022] DELETE /identity/v3/users/33558a40a4984a00bea603e5ae8d68d6 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:26:49.972628 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ef609-f09b-4ae9-8a9f-ee505caf2f0c admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:49.973955 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-078ef609-f09b-4ae9-8a9f-ee505caf2f0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:49.985288 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:50.023364 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-7e52f201-2089-40b9-92a6-96b22f833721 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9869121b0ac648388d9e94273fa70230', 'request_id': 'req-7e52f201-2089-40b9-92a6-96b22f833721'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:50.030963 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-7e52f201-2089-40b9-92a6-96b22f833721 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9869121b0ac648388d9e94273fa70230', 'request_id': 'req-7e52f201-2089-40b9-92a6-96b22f833721'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:50.038154 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-7e52f201-2089-40b9-92a6-96b22f833721 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9869121b0ac648388d9e94273fa70230', 'request_id': 'req-7e52f201-2089-40b9-92a6-96b22f833721'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:50.047062 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ef609-f09b-4ae9-8a9f-ee505caf2f0c admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:50.054938 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-7e52f201-2089-40b9-92a6-96b22f833721 admin admin] The token cache is being invalidate because project 9869121b0ac648388d9e94273fa70230 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:26:50.055236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-7e52f201-2089-40b9-92a6-96b22f833721 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:50.056396 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-7e52f201-2089-40b9-92a6-96b22f833721 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:50.058915 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 966/1922] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:26:49 2022] DELETE /identity/v3/projects/9869121b0ac648388d9e94273fa70230 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:50.060587 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:50.062891 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e52f201-2089-40b9-92a6-96b22f833721 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:50.065076 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-7e52f201-2089-40b9-92a6-96b22f833721 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:50.111704 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d7fac8-6c91-40e3-92d1-913e429e3def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:50.112377 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7fac8-6c91-40e3-92d1-913e429e3def admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:50.112661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7fac8-6c91-40e3-92d1-913e429e3def admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:50.112935 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7fac8-6c91-40e3-92d1-913e429e3def admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:50.113578 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d7fac8-6c91-40e3-92d1-913e429e3def admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:50.117973 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d7fac8-6c91-40e3-92d1-913e429e3def admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:50.145798 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e52f201-2089-40b9-92a6-96b22f833721 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:50.147052 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 957/1923] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:26:49 2022] POST /identity/v3/users => generated 297 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:50.150897 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d7fac8-6c91-40e3-92d1-913e429e3def admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:50.151990 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d94f98-c40a-4e19-b17a-d344fd97b05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:50.152501 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-f8d7fac8-6c91-40e3-92d1-913e429e3def admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:50.153037 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d94f98-c40a-4e19-b17a-d344fd97b05b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:50.153365 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d94f98-c40a-4e19-b17a-d344fd97b05b admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:50.153644 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d94f98-c40a-4e19-b17a-d344fd97b05b admin admin] PATH_INFO: `/v3/users/3d4ffe7637984a9482a81016a8401d93` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:50.154878 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d94f98-c40a-4e19-b17a-d344fd97b05b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d4ffe7637984a9482a81016a8401d93)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:50.156804 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d94f98-c40a-4e19-b17a-d344fd97b05b admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:50.164818 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d7fac8-6c91-40e3-92d1-913e429e3def admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:50.170387 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3011e9-d194-4e91-bbbc-b8296722dc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:50.171085 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3011e9-d194-4e91-bbbc-b8296722dc7f admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:50.171364 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3011e9-d194-4e91-bbbc-b8296722dc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:50.171635 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3011e9-d194-4e91-bbbc-b8296722dc7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:50.172283 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3011e9-d194-4e91-bbbc-b8296722dc7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:50.173875 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3011e9-d194-4e91-bbbc-b8296722dc7f admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:50.186520 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 958/1924] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 14:26:50.190851 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3011e9-d194-4e91-bbbc-b8296722dc7f admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:50.201600 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3011e9-d194-4e91-bbbc-b8296722dc7f admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:50.208315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e060c88d-7021-41a5-ae76-99ef3f378cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:50.209033 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060c88d-7021-41a5-ae76-99ef3f378cc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:50.209355 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060c88d-7021-41a5-ae76-99ef3f378cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:50.209625 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060c88d-7021-41a5-ae76-99ef3f378cc8 admin admin] PATH_INFO: `/v3/projects/adf0cc68cb1245e3a867cf6dcee4b882/users/4226b1d860a1495dabd423c98065370e/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:50.216150 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-02d94f98-c40a-4e19-b17a-d344fd97b05b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d4ffe7637984a9482a81016a8401d93'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:50.216892 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-02d94f98-c40a-4e19-b17a-d344fd97b05b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d4ffe7637984a9482a81016a8401d93'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:50.217511 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-02d94f98-c40a-4e19-b17a-d344fd97b05b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d4ffe7637984a9482a81016a8401d93'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:50.226223 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e060c88d-7021-41a5-ae76-99ef3f378cc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adf0cc68cb1245e3a867cf6dcee4b882, user_id=4226b1d860a1495dabd423c98065370e, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:50.229093 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e060c88d-7021-41a5-ae76-99ef3f378cc8 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:50.234948 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-02d94f98-c40a-4e19-b17a-d344fd97b05b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d4ffe7637984a9482a81016a8401d93'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:50.245238 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 959/1925] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:50 2022] PUT /identity/v3/projects/adf0cc68cb1245e3a867cf6dcee4b882/users/4226b1d860a1495dabd423c98065370e/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:26:50.249934 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e060c88d-7021-41a5-ae76-99ef3f378cc8 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:50.250224 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-02d94f98-c40a-4e19-b17a-d344fd97b05b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d4ffe7637984a9482a81016a8401d93'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:50.252238 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-e060c88d-7021-41a5-ae76-99ef3f378cc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:50.273907 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 967/1926] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:26:50 2022] DELETE /identity/v3/users/3d4ffe7637984a9482a81016a8401d93 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:50.275000 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d94f98-c40a-4e19-b17a-d344fd97b05b admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:50.278784 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-02d94f98-c40a-4e19-b17a-d344fd97b05b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:50.311980 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:50.336837 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e060c88d-7021-41a5-ae76-99ef3f378cc8 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:50.363615 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d94f98-c40a-4e19-b17a-d344fd97b05b admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:50.369686 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd0d703-dec7-4b15-8a6b-358c42352d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:50.370334 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0d703-dec7-4b15-8a6b-358c42352d9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:50.370613 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0d703-dec7-4b15-8a6b-358c42352d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:50.370917 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0d703-dec7-4b15-8a6b-358c42352d9d admin admin] PATH_INFO: `/v3/projects/d25245b7fe7e4beba7f9e19bbca30b19` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:50.372044 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd0d703-dec7-4b15-8a6b-358c42352d9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d25245b7fe7e4beba7f9e19bbca30b19)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:50.374082 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd0d703-dec7-4b15-8a6b-358c42352d9d admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:50.420411 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b4506-dfb0-440d-b48c-7a656b00e36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:50.421257 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b4506-dfb0-440d-b48c-7a656b00e36c admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:50.421932 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b4506-dfb0-440d-b48c-7a656b00e36c admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:50.422442 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b4506-dfb0-440d-b48c-7a656b00e36c admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:50.423072 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62b4506-dfb0-440d-b48c-7a656b00e36c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:50.424973 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62b4506-dfb0-440d-b48c-7a656b00e36c admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:50.435233 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9dd0d703-dec7-4b15-8a6b-358c42352d9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd25245b7fe7e4beba7f9e19bbca30b19', 'request_id': 'req-9dd0d703-dec7-4b15-8a6b-358c42352d9d'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:50.438232 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 960/1927] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:50.441721 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9dd0d703-dec7-4b15-8a6b-358c42352d9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd25245b7fe7e4beba7f9e19bbca30b19', 'request_id': 'req-9dd0d703-dec7-4b15-8a6b-358c42352d9d'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:50.444380 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b4506-dfb0-440d-b48c-7a656b00e36c admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:50.451633 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9dd0d703-dec7-4b15-8a6b-358c42352d9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd25245b7fe7e4beba7f9e19bbca30b19', 'request_id': 'req-9dd0d703-dec7-4b15-8a6b-358c42352d9d'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:50.455997 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b4506-dfb0-440d-b48c-7a656b00e36c admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:50.470769 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:50.471222 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9dd0d703-dec7-4b15-8a6b-358c42352d9d admin admin] The token cache is being invalidate because project d25245b7fe7e4beba7f9e19bbca30b19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:26:50.472332 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9dd0d703-dec7-4b15-8a6b-358c42352d9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:50.473192 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9dd0d703-dec7-4b15-8a6b-358c42352d9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:50.475695 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 968/1928] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:26:50 2022] DELETE /identity/v3/projects/d25245b7fe7e4beba7f9e19bbca30b19 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:50.481130 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6b9101-8944-4bf6-815e-4fbfd06ad9d4 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:50.481412 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6b9101-8944-4bf6-815e-4fbfd06ad9d4 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:50.481684 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6b9101-8944-4bf6-815e-4fbfd06ad9d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:50.511809 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-5d6b9101-8944-4bf6-815e-4fbfd06ad9d4 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:50.513341 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-5d6b9101-8944-4bf6-815e-4fbfd06ad9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:50.527596 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892d53f-e2c3-4b3f-8e24-2acc087cebf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:50.528439 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892d53f-e2c3-4b3f-8e24-2acc087cebf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:50.530333 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892d53f-e2c3-4b3f-8e24-2acc087cebf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:50.530739 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892d53f-e2c3-4b3f-8e24-2acc087cebf2 admin admin] PATH_INFO: `/v3/projects/adf0cc68cb1245e3a867cf6dcee4b882/users/4226b1d860a1495dabd423c98065370e/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:50.532969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9892d53f-e2c3-4b3f-8e24-2acc087cebf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adf0cc68cb1245e3a867cf6dcee4b882, user_id=4226b1d860a1495dabd423c98065370e, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:50.536280 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9892d53f-e2c3-4b3f-8e24-2acc087cebf2 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:50.549669 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 961/1929] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:50 2022] PUT /identity/v3/projects/adf0cc68cb1245e3a867cf6dcee4b882/users/4226b1d860a1495dabd423c98065370e/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:50.563333 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62226f5b-1674-414f-88a1-0da3ac064718 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:50.563658 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62226f5b-1674-414f-88a1-0da3ac064718 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:50.563896 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62226f5b-1674-414f-88a1-0da3ac064718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:50.616845 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 969/1930] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:50.626073 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68164976-057b-4983-b816-da9e714c208f None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:50.626929 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68164976-057b-4983-b816-da9e714c208f None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:50.627530 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68164976-057b-4983-b816-da9e714c208f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:50.628898 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-62226f5b-1674-414f-88a1-0da3ac064718 None None] MFA Rules not processed for user `4226b1d860a1495dabd423c98065370e`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:50.630162 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-62226f5b-1674-414f-88a1-0da3ac064718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:50.652144 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-68164976-057b-4983-b816-da9e714c208f None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:50.653301 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-68164976-057b-4983-b816-da9e714c208f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:50.717297 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:50.738858 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 970/1931] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:50.742795 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68164976-057b-4983-b816-da9e714c208f None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:50.744442 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-68164976-057b-4983-b816-da9e714c208f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:50.748707 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 962/1932] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:50 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:50.751085 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62226f5b-1674-414f-88a1-0da3ac064718 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:50.757983 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68164976-057b-4983-b816-da9e714c208f None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:50.763138 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9149190-d049-42e0-9bd9-cd073b8b1dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:50.763773 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9149190-d049-42e0-9bd9-cd073b8b1dd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:50.764051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9149190-d049-42e0-9bd9-cd073b8b1dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:50.764320 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9149190-d049-42e0-9bd9-cd073b8b1dd0 admin admin] PATH_INFO: `/v3/projects/80101e88df744193b7685bcea9b4f97b` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:50.765449 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9149190-d049-42e0-9bd9-cd073b8b1dd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80101e88df744193b7685bcea9b4f97b)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:50.767560 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9149190-d049-42e0-9bd9-cd073b8b1dd0 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:50.770131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62226f5b-1674-414f-88a1-0da3ac064718 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:50.777655 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc7cc97-85c4-4f05-bb1d-c8cd72a6d6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:50.779569 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7cc97-85c4-4f05-bb1d-c8cd72a6d6e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:50.780831 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7cc97-85c4-4f05-bb1d-c8cd72a6d6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:50.781107 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7cc97-85c4-4f05-bb1d-c8cd72a6d6e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:50.781904 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc7cc97-85c4-4f05-bb1d-c8cd72a6d6e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:26:50.782210 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc7cc97-85c4-4f05-bb1d-c8cd72a6d6e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:50.784285 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc7cc97-85c4-4f05-bb1d-c8cd72a6d6e1 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:50.799807 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 963/1933] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:26:50.800957 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc7cc97-85c4-4f05-bb1d-c8cd72a6d6e1 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:50.802743 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-6dc7cc97-85c4-4f05-bb1d-c8cd72a6d6e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:50.843764 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-b9149190-d049-42e0-9bd9-cd073b8b1dd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80101e88df744193b7685bcea9b4f97b', 'request_id': 'req-b9149190-d049-42e0-9bd9-cd073b8b1dd0'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:50.850768 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-b9149190-d049-42e0-9bd9-cd073b8b1dd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80101e88df744193b7685bcea9b4f97b', 'request_id': 'req-b9149190-d049-42e0-9bd9-cd073b8b1dd0'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:50.856688 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-b9149190-d049-42e0-9bd9-cd073b8b1dd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80101e88df744193b7685bcea9b4f97b', 'request_id': 'req-b9149190-d049-42e0-9bd9-cd073b8b1dd0'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:50.871377 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-b9149190-d049-42e0-9bd9-cd073b8b1dd0 admin admin] The token cache is being invalidate because project 80101e88df744193b7685bcea9b4f97b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:26:50.871671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-b9149190-d049-42e0-9bd9-cd073b8b1dd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:50.872410 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-b9149190-d049-42e0-9bd9-cd073b8b1dd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:26:50.874191 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 971/1934] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:26:50 2022] DELETE /identity/v3/projects/80101e88df744193b7685bcea9b4f97b => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:50.874757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9149190-d049-42e0-9bd9-cd073b8b1dd0 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:50.876117 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-b9149190-d049-42e0-9bd9-cd073b8b1dd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:50.881831 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc7cc97-85c4-4f05-bb1d-c8cd72a6d6e1 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:50.918347 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:50.942603 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ffdecc-3826-4736-a04e-d3eae597024e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:50.944823 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ffdecc-3826-4736-a04e-d3eae597024e service cinder] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:50.945334 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ffdecc-3826-4736-a04e-d3eae597024e service cinder] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:50.945613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ffdecc-3826-4736-a04e-d3eae597024e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:50.947864 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-a1ffdecc-3826-4736-a04e-d3eae597024e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:50.948595 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9149190-d049-42e0-9bd9-cd073b8b1dd0 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:50.953332 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d9ec06-3324-4877-a03f-e925761a1a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:50.953946 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9ec06-3324-4877-a03f-e925761a1a77 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:50.954351 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9ec06-3324-4877-a03f-e925761a1a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:50.954626 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9ec06-3324-4877-a03f-e925761a1a77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:50.955289 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d9ec06-3324-4877-a03f-e925761a1a77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:50.957243 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d9ec06-3324-4877-a03f-e925761a1a77 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:50.990868 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 972/1935] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:26:50 2022] POST /identity/v3/projects => generated 383 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:50.990868 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463d2cf-782f-4fb9-9485-3d45c1202358 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:50.990868 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463d2cf-782f-4fb9-9485-3d45c1202358 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:50.991266 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463d2cf-782f-4fb9-9485-3d45c1202358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:51.011940 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-0463d2cf-782f-4fb9-9485-3d45c1202358 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:51.013307 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-0463d2cf-782f-4fb9-9485-3d45c1202358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:51.028034 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ffdecc-3826-4736-a04e-d3eae597024e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:51.029520 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ffdecc-3826-4736-a04e-d3eae597024e service cinder] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:51.040164 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 964/1936] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:50 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:51.040392 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ffdecc-3826-4736-a04e-d3eae597024e service cinder] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:51.053334 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ffdecc-3826-4736-a04e-d3eae597024e service cinder] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:51.083445 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 973/1937] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:51.085678 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963a4d3-581a-441b-ba02-1bea1b54f987 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:51.086039 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963a4d3-581a-441b-ba02-1bea1b54f987 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:51.086386 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963a4d3-581a-441b-ba02-1bea1b54f987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:51.095316 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:51.108840 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-e963a4d3-581a-441b-ba02-1bea1b54f987 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:51.110301 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-e963a4d3-581a-441b-ba02-1bea1b54f987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:51.122475 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 974/1938] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:51.123835 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963a4d3-581a-441b-ba02-1bea1b54f987 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:51.136479 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963a4d3-581a-441b-ba02-1bea1b54f987 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:51.141513 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ff3b1-6414-4d29-9a9e-6a6df3987a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:51.142194 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ff3b1-6414-4d29-9a9e-6a6df3987a84 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:51.142536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ff3b1-6414-4d29-9a9e-6a6df3987a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:51.142900 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ff3b1-6414-4d29-9a9e-6a6df3987a84 admin admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:51.143677 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5ff3b1-6414-4d29-9a9e-6a6df3987a84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:26:51.144039 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5ff3b1-6414-4d29-9a9e-6a6df3987a84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:51.144575 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd8b8a-43be-45e0-b705-ec80adcdce8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:51.145379 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd8b8a-43be-45e0-b705-ec80adcdce8b admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:51.145824 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd8b8a-43be-45e0-b705-ec80adcdce8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:51.146141 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5ff3b1-6414-4d29-9a9e-6a6df3987a84 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:51.146261 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd8b8a-43be-45e0-b705-ec80adcdce8b admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:51.147525 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fd8b8a-43be-45e0-b705-ec80adcdce8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:51.149480 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fd8b8a-43be-45e0-b705-ec80adcdce8b admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:51.160248 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 975/1939] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:26:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:26:51.161954 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ff3b1-6414-4d29-9a9e-6a6df3987a84 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:51.164774 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-fa5ff3b1-6414-4d29-9a9e-6a6df3987a84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:51.174865 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 965/1940] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:26:51 2022] POST /identity/v3/users => generated 292 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:51.174865 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd8b8a-43be-45e0-b705-ec80adcdce8b admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:51.182764 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd8b8a-43be-45e0-b705-ec80adcdce8b admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:51.187339 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35979a4d-110b-4f32-b8e6-1d7051a65d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:51.187919 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35979a4d-110b-4f32-b8e6-1d7051a65d4c admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:51.188198 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35979a4d-110b-4f32-b8e6-1d7051a65d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:51.188491 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35979a4d-110b-4f32-b8e6-1d7051a65d4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:51.189134 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35979a4d-110b-4f32-b8e6-1d7051a65d4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:51.191076 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35979a4d-110b-4f32-b8e6-1d7051a65d4c admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:51.219251 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 966/1941] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:26:51 2022] POST /identity/v3/projects => generated 391 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:51.220070 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35979a4d-110b-4f32-b8e6-1d7051a65d4c admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:51.224563 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ff3b1-6414-4d29-9a9e-6a6df3987a84 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:51.278237 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f4df3-0ad6-43d9-9ee8-01581e71669b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:51.278949 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f4df3-0ad6-43d9-9ee8-01581e71669b service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:51.279236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f4df3-0ad6-43d9-9ee8-01581e71669b service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:51.279583 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f4df3-0ad6-43d9-9ee8-01581e71669b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:51.280035 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:51.281445 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-449f4df3-0ad6-43d9-9ee8-01581e71669b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:51.286601 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35979a4d-110b-4f32-b8e6-1d7051a65d4c admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:51.292683 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13225172-92d4-4710-b17f-070aa9a73493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:51.293289 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13225172-92d4-4710-b17f-070aa9a73493 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:51.293569 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13225172-92d4-4710-b17f-070aa9a73493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:51.293839 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13225172-92d4-4710-b17f-070aa9a73493 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:51.294480 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13225172-92d4-4710-b17f-070aa9a73493 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:51.296119 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13225172-92d4-4710-b17f-070aa9a73493 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:51.306210 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 967/1942] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:51.307032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13225172-92d4-4710-b17f-070aa9a73493 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:51.316229 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13225172-92d4-4710-b17f-070aa9a73493 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:51.321626 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8ecfe-c11c-4e27-a327-4d5d9cf56e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:51.322358 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8ecfe-c11c-4e27-a327-4d5d9cf56e1f admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:51.322636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8ecfe-c11c-4e27-a327-4d5d9cf56e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:51.322954 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8ecfe-c11c-4e27-a327-4d5d9cf56e1f admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:51.323602 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab8ecfe-c11c-4e27-a327-4d5d9cf56e1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:51.325490 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab8ecfe-c11c-4e27-a327-4d5d9cf56e1f admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:51.346969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 968/1943] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:26:51 2022] POST /identity/v3/users => generated 296 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:51.347777 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8ecfe-c11c-4e27-a327-4d5d9cf56e1f admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:51.352947 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449f4df3-0ad6-43d9-9ee8-01581e71669b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:51.354432 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449f4df3-0ad6-43d9-9ee8-01581e71669b service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:51.357060 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8ecfe-c11c-4e27-a327-4d5d9cf56e1f admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:51.361680 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b3714-cfc3-4991-b4f9-7d286298635e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:51.362199 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b3714-cfc3-4991-b4f9-7d286298635e admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:51.362479 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b3714-cfc3-4991-b4f9-7d286298635e admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:51.362748 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b3714-cfc3-4991-b4f9-7d286298635e admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:51.363409 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b3714-cfc3-4991-b4f9-7d286298635e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:51.363740 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 976/1944] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:51 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:26:51.364642 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f4df3-0ad6-43d9-9ee8-01581e71669b service cinder] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:51.364984 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b3714-cfc3-4991-b4f9-7d286298635e admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:51.376383 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f4df3-0ad6-43d9-9ee8-01581e71669b service cinder] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:51.378996 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 969/1945] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:51.380476 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b3714-cfc3-4991-b4f9-7d286298635e admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:51.382723 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af8333-930b-4654-bed1-02804a07994c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:51.383476 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af8333-930b-4654-bed1-02804a07994c admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:51.383755 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af8333-930b-4654-bed1-02804a07994c admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:51.384026 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af8333-930b-4654-bed1-02804a07994c admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:51.384695 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75af8333-930b-4654-bed1-02804a07994c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:51.386574 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75af8333-930b-4654-bed1-02804a07994c admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:51.391704 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b3714-cfc3-4991-b4f9-7d286298635e admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:51.399959 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 977/1946] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:51.400262 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51809a3f-0904-4c6f-a661-06941fcd2df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:51.400262 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51809a3f-0904-4c6f-a661-06941fcd2df2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:51.400262 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51809a3f-0904-4c6f-a661-06941fcd2df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:51.400262 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51809a3f-0904-4c6f-a661-06941fcd2df2 admin admin] PATH_INFO: `/v3/projects/fcc17c3a205b441db38b2e70152cdd35/users/e77d048ffd33407aae0dbbd04e2c30ea/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:51.400950 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af8333-930b-4654-bed1-02804a07994c admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:51.412623 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af8333-930b-4654-bed1-02804a07994c admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:51.416146 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51809a3f-0904-4c6f-a661-06941fcd2df2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcc17c3a205b441db38b2e70152cdd35, user_id=e77d048ffd33407aae0dbbd04e2c30ea, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:51.417494 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7320b0e9-314b-4976-8c35-3e17f4e1e25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:51.418171 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320b0e9-314b-4976-8c35-3e17f4e1e25c admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:51.418463 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320b0e9-314b-4976-8c35-3e17f4e1e25c admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:51.418649 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320b0e9-314b-4976-8c35-3e17f4e1e25c admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:51.419020 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51809a3f-0904-4c6f-a661-06941fcd2df2 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:51.419282 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7320b0e9-314b-4976-8c35-3e17f4e1e25c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:51.420834 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7320b0e9-314b-4976-8c35-3e17f4e1e25c admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:51.436155 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 978/1947] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:51.436740 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 970/1948] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:51 2022] PUT /identity/v3/projects/fcc17c3a205b441db38b2e70152cdd35/users/e77d048ffd33407aae0dbbd04e2c30ea/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:26:51.438117 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7320b0e9-314b-4976-8c35-3e17f4e1e25c admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:51.446499 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e480ff-c1f7-4c2e-a147-21a9b164b298 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:51.446701 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e480ff-c1f7-4c2e-a147-21a9b164b298 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:51.447127 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e480ff-c1f7-4c2e-a147-21a9b164b298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:51.461636 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7320b0e9-314b-4976-8c35-3e17f4e1e25c admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:51.476618 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-99e480ff-c1f7-4c2e-a147-21a9b164b298 None None] MFA Rules not processed for user `e77d048ffd33407aae0dbbd04e2c30ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:51.478082 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-99e480ff-c1f7-4c2e-a147-21a9b164b298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:51.495498 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:51.538199 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b7d83-17bd-4cb9-aae0-3dc3b05376fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:51.538908 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b7d83-17bd-4cb9-aae0-3dc3b05376fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:51.539187 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b7d83-17bd-4cb9-aae0-3dc3b05376fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:51.539461 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b7d83-17bd-4cb9-aae0-3dc3b05376fd admin admin] PATH_INFO: `/v3/projects/8a1054585ddc4cc39a4f5e721b8f240c/users/acf4bea68e9b4970967026cdcda671e1/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:51.549698 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 971/1949] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:51.555105 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e480ff-c1f7-4c2e-a147-21a9b164b298 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:51.556292 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15b7d83-17bd-4cb9-aae0-3dc3b05376fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a1054585ddc4cc39a4f5e721b8f240c, user_id=acf4bea68e9b4970967026cdcda671e1, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:51.559036 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15b7d83-17bd-4cb9-aae0-3dc3b05376fd admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:51.567522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e480ff-c1f7-4c2e-a147-21a9b164b298 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:51.574276 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 979/1950] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:51 2022] PUT /identity/v3/projects/8a1054585ddc4cc39a4f5e721b8f240c/users/acf4bea68e9b4970967026cdcda671e1/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:26:51.587265 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d59b3-1d49-4315-92ff-6aa39b1baa01 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:51.587434 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d59b3-1d49-4315-92ff-6aa39b1baa01 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:51.587869 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d59b3-1d49-4315-92ff-6aa39b1baa01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:51.628083 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-063d59b3-1d49-4315-92ff-6aa39b1baa01 None None] MFA Rules not processed for user `acf4bea68e9b4970967026cdcda671e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:51.629453 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-063d59b3-1d49-4315-92ff-6aa39b1baa01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:51.636281 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5cd41-2f86-427c-b538-79285f4bb583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:51.636888 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5cd41-2f86-427c-b538-79285f4bb583 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:51.637167 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5cd41-2f86-427c-b538-79285f4bb583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:51.637438 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5cd41-2f86-427c-b538-79285f4bb583 admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:51.638122 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa5cd41-2f86-427c-b538-79285f4bb583 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:51.640938 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa5cd41-2f86-427c-b538-79285f4bb583 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:51.683954 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 972/1951] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:26:51 2022] POST /identity/v3/projects => generated 383 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:51.688991 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5cd41-2f86-427c-b538-79285f4bb583 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:51.705116 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:51.711598 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 980/1952] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:51 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:26:51.716167 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063d59b3-1d49-4315-92ff-6aa39b1baa01 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:51.731272 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063d59b3-1d49-4315-92ff-6aa39b1baa01 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:51.780300 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5cd41-2f86-427c-b538-79285f4bb583 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:51.786696 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25effc81-d7b7-48b1-b05d-fb8122384577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:51.788065 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25effc81-d7b7-48b1-b05d-fb8122384577 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:51.788497 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25effc81-d7b7-48b1-b05d-fb8122384577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:51.788918 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25effc81-d7b7-48b1-b05d-fb8122384577 admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:51.789587 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25effc81-d7b7-48b1-b05d-fb8122384577 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:51.791573 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25effc81-d7b7-48b1-b05d-fb8122384577 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:51.810211 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa4181c-01ee-4120-a95b-779a28522649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:51.810894 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4181c-01ee-4120-a95b-779a28522649 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:51.811362 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4181c-01ee-4120-a95b-779a28522649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:51.811591 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4181c-01ee-4120-a95b-779a28522649 admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:51.812278 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa4181c-01ee-4120-a95b-779a28522649 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:51.818717 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa4181c-01ee-4120-a95b-779a28522649 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:51.828619 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 973/1953] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:26:51 2022] POST /identity/v3/users => generated 298 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:51.834711 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25effc81-d7b7-48b1-b05d-fb8122384577 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:51.859256 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25effc81-d7b7-48b1-b05d-fb8122384577 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:51.864764 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6942cc16-f369-4f1f-965e-bd63c36d7b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:51.865927 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942cc16-f369-4f1f-965e-bd63c36d7b05 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:51.866215 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942cc16-f369-4f1f-965e-bd63c36d7b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:51.866486 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942cc16-f369-4f1f-965e-bd63c36d7b05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:51.867223 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6942cc16-f369-4f1f-965e-bd63c36d7b05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:51.869447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6942cc16-f369-4f1f-965e-bd63c36d7b05 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:51.872347 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 981/1954] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:26:51 2022] POST /identity/v3/projects => generated 389 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:26:51.876529 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa4181c-01ee-4120-a95b-779a28522649 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:51.889808 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 974/1955] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:51.890734 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6942cc16-f369-4f1f-965e-bd63c36d7b05 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:51.902804 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6942cc16-f369-4f1f-965e-bd63c36d7b05 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:51.923614 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:51.975216 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd1b714-ebc3-420f-a1cd-b7087bb7eeaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:51.975739 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa4181c-01ee-4120-a95b-779a28522649 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:51.976381 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1b714-ebc3-420f-a1cd-b7087bb7eeaf admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:51.976381 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1b714-ebc3-420f-a1cd-b7087bb7eeaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:51.976599 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1b714-ebc3-420f-a1cd-b7087bb7eeaf admin admin] PATH_INFO: `/v3/projects/f75af7d106f24083af134d63da0dcf3d/users/cdc6fa57cc7c497b9224dad34bd355e4/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:51.981629 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab796517-c6bf-472d-8df8-a0de72d414c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:51.982240 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab796517-c6bf-472d-8df8-a0de72d414c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:51.982517 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab796517-c6bf-472d-8df8-a0de72d414c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:51.982788 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab796517-c6bf-472d-8df8-a0de72d414c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:51.983448 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab796517-c6bf-472d-8df8-a0de72d414c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:51.985796 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab796517-c6bf-472d-8df8-a0de72d414c7 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:51.994423 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd1b714-ebc3-420f-a1cd-b7087bb7eeaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f75af7d106f24083af134d63da0dcf3d, user_id=cdc6fa57cc7c497b9224dad34bd355e4, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:51.997051 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd1b714-ebc3-420f-a1cd-b7087bb7eeaf admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:52.011396 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 982/1956] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:26:51 2022] POST /identity/v3/users => generated 301 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:26:52.011566 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 975/1957] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:51 2022] PUT /identity/v3/projects/f75af7d106f24083af134d63da0dcf3d/users/cdc6fa57cc7c497b9224dad34bd355e4/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:52.012592 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab796517-c6bf-472d-8df8-a0de72d414c7 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:52.014664 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd1b714-ebc3-420f-a1cd-b7087bb7eeaf admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:52.027500 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab796517-c6bf-472d-8df8-a0de72d414c7 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:52.078345 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd1b714-ebc3-420f-a1cd-b7087bb7eeaf admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:52.084417 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c820dca-95e1-41b0-9fc9-af613d682042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:52.085105 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c820dca-95e1-41b0-9fc9-af613d682042 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:52.085451 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c820dca-95e1-41b0-9fc9-af613d682042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:52.085814 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c820dca-95e1-41b0-9fc9-af613d682042 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:52.087074 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c820dca-95e1-41b0-9fc9-af613d682042 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:52.087858 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc568fd-48c3-47c2-b050-49f4532bb4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:52.088463 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc568fd-48c3-47c2-b050-49f4532bb4f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:52.088685 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c820dca-95e1-41b0-9fc9-af613d682042 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:52.088778 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc568fd-48c3-47c2-b050-49f4532bb4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:52.088991 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc568fd-48c3-47c2-b050-49f4532bb4f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:52.089632 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc568fd-48c3-47c2-b050-49f4532bb4f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:52.091340 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc568fd-48c3-47c2-b050-49f4532bb4f6 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:52.100339 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 976/1958] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:52.102000 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 983/1959] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:52.104399 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c820dca-95e1-41b0-9fc9-af613d682042 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:52.106068 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc568fd-48c3-47c2-b050-49f4532bb4f6 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:52.116091 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c820dca-95e1-41b0-9fc9-af613d682042 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:52.120727 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc568fd-48c3-47c2-b050-49f4532bb4f6 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:52.120853 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebafff5-2a3e-4d32-944b-3839bfcf4732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:52.121567 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebafff5-2a3e-4d32-944b-3839bfcf4732 admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:52.121844 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebafff5-2a3e-4d32-944b-3839bfcf4732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:52.122109 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebafff5-2a3e-4d32-944b-3839bfcf4732 admin admin] PATH_INFO: `/v3/projects/f75af7d106f24083af134d63da0dcf3d/users/cdc6fa57cc7c497b9224dad34bd355e4/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:52.124180 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebafff5-2a3e-4d32-944b-3839bfcf4732 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f75af7d106f24083af134d63da0dcf3d, user_id=cdc6fa57cc7c497b9224dad34bd355e4, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:52.125607 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348a0da0-85cb-4d3f-992e-a42817c2a9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:52.126269 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a0da0-85cb-4d3f-992e-a42817c2a9b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:52.126549 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a0da0-85cb-4d3f-992e-a42817c2a9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:52.126801 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebafff5-2a3e-4d32-944b-3839bfcf4732 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:52.126992 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a0da0-85cb-4d3f-992e-a42817c2a9b1 admin admin] PATH_INFO: `/v3/projects/3c01fb2231f1458db5bc419f3dd66629/users/c83d9c9fc7db4ba2ba1cfe9dc678b3bf/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:52.132612 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:52.138985 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 977/1960] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:52 2022] PUT /identity/v3/projects/f75af7d106f24083af134d63da0dcf3d/users/cdc6fa57cc7c497b9224dad34bd355e4/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:26:52.149924 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f4c298-f162-4af5-81f8-8a4014a762ae None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:52.150329 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348a0da0-85cb-4d3f-992e-a42817c2a9b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c01fb2231f1458db5bc419f3dd66629, user_id=c83d9c9fc7db4ba2ba1cfe9dc678b3bf, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:52.150583 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f4c298-f162-4af5-81f8-8a4014a762ae None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:52.150698 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f4c298-f162-4af5-81f8-8a4014a762ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:52.153341 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348a0da0-85cb-4d3f-992e-a42817c2a9b1 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:52.168978 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 984/1961] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:52 2022] PUT /identity/v3/projects/3c01fb2231f1458db5bc419f3dd66629/users/c83d9c9fc7db4ba2ba1cfe9dc678b3bf/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:26:52.172981 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348a0da0-85cb-4d3f-992e-a42817c2a9b1 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:52.182239 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-76f4c298-f162-4af5-81f8-8a4014a762ae None None] MFA Rules not processed for user `cdc6fa57cc7c497b9224dad34bd355e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:52.183506 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-76f4c298-f162-4af5-81f8-8a4014a762ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:52.248795 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348a0da0-85cb-4d3f-992e-a42817c2a9b1 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:52.253470 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58087f4-ec79-4c8b-8c42-c8f75e4fa844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:52.254071 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58087f4-ec79-4c8b-8c42-c8f75e4fa844 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:52.254347 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58087f4-ec79-4c8b-8c42-c8f75e4fa844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:52.254616 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58087f4-ec79-4c8b-8c42-c8f75e4fa844 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:52.255284 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58087f4-ec79-4c8b-8c42-c8f75e4fa844 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:52.257102 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58087f4-ec79-4c8b-8c42-c8f75e4fa844 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:52.260274 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 978/1962] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:52 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:26:52.269973 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 985/1963] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:26:52.271037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f4c298-f162-4af5-81f8-8a4014a762ae None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:52.273308 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58087f4-ec79-4c8b-8c42-c8f75e4fa844 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:52.275010 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-c58087f4-ec79-4c8b-8c42-c8f75e4fa844 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:52.282217 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f4c298-f162-4af5-81f8-8a4014a762ae None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:52.287652 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e1a6a-f577-4926-a933-0bb36b4caa47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:52.288321 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e1a6a-f577-4926-a933-0bb36b4caa47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:52.288599 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e1a6a-f577-4926-a933-0bb36b4caa47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:52.288876 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e1a6a-f577-4926-a933-0bb36b4caa47 admin admin] PATH_INFO: `/v3/projects/3c01fb2231f1458db5bc419f3dd66629/users/c83d9c9fc7db4ba2ba1cfe9dc678b3bf/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:52.290845 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e1a6a-f577-4926-a933-0bb36b4caa47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c01fb2231f1458db5bc419f3dd66629, user_id=c83d9c9fc7db4ba2ba1cfe9dc678b3bf, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:52.293493 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e1a6a-f577-4926-a933-0bb36b4caa47 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:52.303536 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 979/1964] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:26:52 2022] PUT /identity/v3/projects/3c01fb2231f1458db5bc419f3dd66629/users/c83d9c9fc7db4ba2ba1cfe9dc678b3bf/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:26:52.307634 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:26:52.313982 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05062cdb-6bb7-48ba-91c9-e0235edd6b10 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:52.314262 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05062cdb-6bb7-48ba-91c9-e0235edd6b10 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:52.314529 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05062cdb-6bb7-48ba-91c9-e0235edd6b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:52.335279 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58087f4-ec79-4c8b-8c42-c8f75e4fa844 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:52.342628 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-05062cdb-6bb7-48ba-91c9-e0235edd6b10 None None] MFA Rules not processed for user `c83d9c9fc7db4ba2ba1cfe9dc678b3bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:26:52.343876 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-05062cdb-6bb7-48ba-91c9-e0235edd6b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:26:52.385569 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c265aa72-dae8-4a03-8d2b-ebf80e054d47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:52.386346 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265aa72-dae8-4a03-8d2b-ebf80e054d47 service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:52.386735 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265aa72-dae8-4a03-8d2b-ebf80e054d47 service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:52.387144 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265aa72-dae8-4a03-8d2b-ebf80e054d47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:52.412863 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 980/1965] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:26:52 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:26:52.424417 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05062cdb-6bb7-48ba-91c9-e0235edd6b10 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:52.433522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05062cdb-6bb7-48ba-91c9-e0235edd6b10 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:52.437097 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84026e-15cf-4098-ae6c-772b546b53c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:52.437723 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84026e-15cf-4098-ae6c-772b546b53c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:52.438004 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84026e-15cf-4098-ae6c-772b546b53c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:52.438276 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84026e-15cf-4098-ae6c-772b546b53c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:52.447909 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a84026e-15cf-4098-ae6c-772b546b53c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:52.448031 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c265aa72-dae8-4a03-8d2b-ebf80e054d47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:52.449341 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a84026e-15cf-4098-ae6c-772b546b53c7 service cinder] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:52.449541 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c265aa72-dae8-4a03-8d2b-ebf80e054d47 service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:52.458671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 986/1966] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 14:26:52.459616 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 981/1967] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:52 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 14:26:52.525441 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c265aa72-dae8-4a03-8d2b-ebf80e054d47 service cinder] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:26:52.534969 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c265aa72-dae8-4a03-8d2b-ebf80e054d47 service cinder] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:26:52.538842 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64960aff-9b0d-4091-a9c2-e60056c3c721 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:26:52.539483 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64960aff-9b0d-4091-a9c2-e60056c3c721 service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:26:52.539760 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64960aff-9b0d-4091-a9c2-e60056c3c721 service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:26:52.540030 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64960aff-9b0d-4091-a9c2-e60056c3c721 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:26:52.604240 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64960aff-9b0d-4091-a9c2-e60056c3c721 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:26:52.605795 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64960aff-9b0d-4091-a9c2-e60056c3c721 service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:26:52.616146 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 987/1968] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:26:52 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:27:02.454445 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=80,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:02.460533 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb59b6f-669f-45a6-9495-43698a447eaa None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:02.460892 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb59b6f-669f-45a6-9495-43698a447eaa None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:02.461302 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb59b6f-669f-45a6-9495-43698a447eaa None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:02.464143 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 982/1969] 10.222.0.52 () {54 vars in 921 bytes} [Sat Sep 24 14:27:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:27:02.472739 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e610f2d-a4c4-4a78-b051-c10a8c85ce77 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:02.473279 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e610f2d-a4c4-4a78-b051-c10a8c85ce77 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:02.473279 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e610f2d-a4c4-4a78-b051-c10a8c85ce77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:02.498980 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-4e610f2d-a4c4-4a78-b051-c10a8c85ce77 None None] MFA Rules not processed for user `79eaaae97aac4282a613939e5b768376`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:02.500927 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-4e610f2d-a4c4-4a78-b051-c10a8c85ce77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:02.554133 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:02.596446 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 988/1970] 10.222.0.52 () {58 vars in 1018 bytes} [Sat Sep 24 14:27:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:27:02.609398 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb59b6f-669f-45a6-9495-43698a447eaa None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:02.611420 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-6bb59b6f-669f-45a6-9495-43698a447eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:02.631401 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb59b6f-669f-45a6-9495-43698a447eaa None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:02.645031 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5068d973-78b9-46c7-9f38-e17b4299af20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:02.645660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5068d973-78b9-46c7-9f38-e17b4299af20 service nova] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:02.645939 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5068d973-78b9-46c7-9f38-e17b4299af20 service nova] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:02.646213 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5068d973-78b9-46c7-9f38-e17b4299af20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:02.656376 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5068d973-78b9-46c7-9f38-e17b4299af20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:02.658395 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5068d973-78b9-46c7-9f38-e17b4299af20 service nova] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:02.676894 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 983/1971] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:27:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:27:09.260709 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e610f2d-a4c4-4a78-b051-c10a8c85ce77 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:09.262180 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-4e610f2d-a4c4-4a78-b051-c10a8c85ce77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:09.429281 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e610f2d-a4c4-4a78-b051-c10a8c85ce77 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:09.469708 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40508549-1d70-4eb6-85f2-4bfb69c2cc7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:09.471188 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40508549-1d70-4eb6-85f2-4bfb69c2cc7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:09.472077 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40508549-1d70-4eb6-85f2-4bfb69c2cc7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:09.473267 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40508549-1d70-4eb6-85f2-4bfb69c2cc7d admin admin] PATH_INFO: `/v3/users/33084eda6b6045d3a031f452d1a05b95` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:09.475760 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40508549-1d70-4eb6-85f2-4bfb69c2cc7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=33084eda6b6045d3a031f452d1a05b95)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:09.478704 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40508549-1d70-4eb6-85f2-4bfb69c2cc7d admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:09.550653 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-40508549-1d70-4eb6-85f2-4bfb69c2cc7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33084eda6b6045d3a031f452d1a05b95'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:09.551408 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-40508549-1d70-4eb6-85f2-4bfb69c2cc7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33084eda6b6045d3a031f452d1a05b95'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:09.552051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-40508549-1d70-4eb6-85f2-4bfb69c2cc7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33084eda6b6045d3a031f452d1a05b95'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:09.570914 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-40508549-1d70-4eb6-85f2-4bfb69c2cc7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33084eda6b6045d3a031f452d1a05b95'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:09.581752 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-40508549-1d70-4eb6-85f2-4bfb69c2cc7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33084eda6b6045d3a031f452d1a05b95'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:09.614344 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 989/1972] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:27:09 2022] DELETE /identity/v3/users/33084eda6b6045d3a031f452d1a05b95 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:09.617895 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be947d6-05c3-47e0-b88f-bc7864427e91 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:09.618378 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be947d6-05c3-47e0-b88f-bc7864427e91 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:09.618733 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be947d6-05c3-47e0-b88f-bc7864427e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:09.647486 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-3be947d6-05c3-47e0-b88f-bc7864427e91 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:09.647486 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-3be947d6-05c3-47e0-b88f-bc7864427e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:09.746390 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 984/1973] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:27:09.760341 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40508549-1d70-4eb6-85f2-4bfb69c2cc7d admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:09.762037 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-40508549-1d70-4eb6-85f2-4bfb69c2cc7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:09.898529 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:09.905256 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40508549-1d70-4eb6-85f2-4bfb69c2cc7d admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:09.918780 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f2d17-931e-4576-8a13-69c1c6a8cac4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:09.919730 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f2d17-931e-4576-8a13-69c1c6a8cac4 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:09.920082 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f2d17-931e-4576-8a13-69c1c6a8cac4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:09.920882 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f2d17-931e-4576-8a13-69c1c6a8cac4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:09.935757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f2d17-931e-4576-8a13-69c1c6a8cac4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:09.937463 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f2d17-931e-4576-8a13-69c1c6a8cac4 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:09.950081 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 990/1974] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:27:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:27:10.065899 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be947d6-05c3-47e0-b88f-bc7864427e91 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:10.067719 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-3be947d6-05c3-47e0-b88f-bc7864427e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:10.082800 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be947d6-05c3-47e0-b88f-bc7864427e91 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:10.094929 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96218a1-fe4d-4bc1-b837-8f098b1a981c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:10.104047 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96218a1-fe4d-4bc1-b837-8f098b1a981c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:10.104519 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96218a1-fe4d-4bc1-b837-8f098b1a981c admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:10.106427 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96218a1-fe4d-4bc1-b837-8f098b1a981c admin admin] PATH_INFO: `/v3/users/acf4bea68e9b4970967026cdcda671e1` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:10.108511 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96218a1-fe4d-4bc1-b837-8f098b1a981c admin admin] RBAC: Authorizing `identity:delete_user(user_id=acf4bea68e9b4970967026cdcda671e1)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:10.114567 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96218a1-fe4d-4bc1-b837-8f098b1a981c admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:10.185880 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-b96218a1-fe4d-4bc1-b837-8f098b1a981c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acf4bea68e9b4970967026cdcda671e1'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:10.186530 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-b96218a1-fe4d-4bc1-b837-8f098b1a981c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acf4bea68e9b4970967026cdcda671e1'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:10.194178 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-b96218a1-fe4d-4bc1-b837-8f098b1a981c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acf4bea68e9b4970967026cdcda671e1'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:10.194889 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-b96218a1-fe4d-4bc1-b837-8f098b1a981c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acf4bea68e9b4970967026cdcda671e1'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:10.207243 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-b96218a1-fe4d-4bc1-b837-8f098b1a981c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acf4bea68e9b4970967026cdcda671e1'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:10.221156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 985/1975] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:27:10 2022] DELETE /identity/v3/users/acf4bea68e9b4970967026cdcda671e1 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:10.226395 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76365d4-ceef-4821-8e57-673402e79ea1 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:10.226684 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76365d4-ceef-4821-8e57-673402e79ea1 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:10.227112 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76365d4-ceef-4821-8e57-673402e79ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:10.246610 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-c76365d4-ceef-4821-8e57-673402e79ea1 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:10.247968 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-c76365d4-ceef-4821-8e57-673402e79ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:10.258200 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96218a1-fe4d-4bc1-b837-8f098b1a981c admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:10.259630 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-b96218a1-fe4d-4bc1-b837-8f098b1a981c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:10.323510 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 991/1976] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:27:10.333445 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76365d4-ceef-4821-8e57-673402e79ea1 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:10.335442 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-c76365d4-ceef-4821-8e57-673402e79ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:10.335592 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96218a1-fe4d-4bc1-b837-8f098b1a981c admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:10.341984 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5df76f-131e-446d-8709-28a4e587e1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:10.342628 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5df76f-131e-446d-8709-28a4e587e1dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:10.342954 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5df76f-131e-446d-8709-28a4e587e1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:10.343234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5df76f-131e-446d-8709-28a4e587e1dd admin admin] PATH_INFO: `/v3/projects/9a33b93133414161a03bde9c002a7e3e` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:10.344370 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5df76f-131e-446d-8709-28a4e587e1dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a33b93133414161a03bde9c002a7e3e)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:10.346318 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5df76f-131e-446d-8709-28a4e587e1dd admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:10.347949 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:10.396615 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76365d4-ceef-4821-8e57-673402e79ea1 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:10.412339 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-7e5df76f-131e-446d-8709-28a4e587e1dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a33b93133414161a03bde9c002a7e3e', 'request_id': 'req-7e5df76f-131e-446d-8709-28a4e587e1dd'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:10.419978 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-7e5df76f-131e-446d-8709-28a4e587e1dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a33b93133414161a03bde9c002a7e3e', 'request_id': 'req-7e5df76f-131e-446d-8709-28a4e587e1dd'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:10.426335 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-7e5df76f-131e-446d-8709-28a4e587e1dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a33b93133414161a03bde9c002a7e3e', 'request_id': 'req-7e5df76f-131e-446d-8709-28a4e587e1dd'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:10.440214 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-7e5df76f-131e-446d-8709-28a4e587e1dd admin admin] The token cache is being invalidate because project 9a33b93133414161a03bde9c002a7e3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:27:10.440499 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-7e5df76f-131e-446d-8709-28a4e587e1dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:10.441807 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-7e5df76f-131e-446d-8709-28a4e587e1dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:10.443590 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 986/1977] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:27:10 2022] DELETE /identity/v3/projects/9a33b93133414161a03bde9c002a7e3e => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:27:10.448032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5df76f-131e-446d-8709-28a4e587e1dd admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:10.449641 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-7e5df76f-131e-446d-8709-28a4e587e1dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:10.463604 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabda12c-4a09-4bf5-a89e-91c96baa8015 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:10.464225 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabda12c-4a09-4bf5-a89e-91c96baa8015 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:10.464502 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabda12c-4a09-4bf5-a89e-91c96baa8015 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:10.464772 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabda12c-4a09-4bf5-a89e-91c96baa8015 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:10.466530 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-aabda12c-4a09-4bf5-a89e-91c96baa8015 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:10.533783 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5df76f-131e-446d-8709-28a4e587e1dd admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:10.538411 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c36cbc-1b05-4eb5-af9e-6041b6bb9fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:10.539059 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c36cbc-1b05-4eb5-af9e-6041b6bb9fab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:10.539342 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c36cbc-1b05-4eb5-af9e-6041b6bb9fab admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:10.539609 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c36cbc-1b05-4eb5-af9e-6041b6bb9fab admin admin] PATH_INFO: `/v3/users/4226b1d860a1495dabd423c98065370e` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:10.540847 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c36cbc-1b05-4eb5-af9e-6041b6bb9fab admin admin] RBAC: Authorizing `identity:delete_user(user_id=4226b1d860a1495dabd423c98065370e)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:10.542874 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c36cbc-1b05-4eb5-af9e-6041b6bb9fab admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:10.547858 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabda12c-4a09-4bf5-a89e-91c96baa8015 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:10.549286 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabda12c-4a09-4bf5-a89e-91c96baa8015 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:10.553212 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:10.562146 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 992/1978] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:27:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:27:10.598923 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-a1c36cbc-1b05-4eb5-af9e-6041b6bb9fab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4226b1d860a1495dabd423c98065370e'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:10.600189 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-a1c36cbc-1b05-4eb5-af9e-6041b6bb9fab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4226b1d860a1495dabd423c98065370e'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:10.608101 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-a1c36cbc-1b05-4eb5-af9e-6041b6bb9fab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4226b1d860a1495dabd423c98065370e'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:10.608791 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-a1c36cbc-1b05-4eb5-af9e-6041b6bb9fab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4226b1d860a1495dabd423c98065370e'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:10.622297 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-a1c36cbc-1b05-4eb5-af9e-6041b6bb9fab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4226b1d860a1495dabd423c98065370e'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:10.638619 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 987/1979] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:27:10 2022] DELETE /identity/v3/users/4226b1d860a1495dabd423c98065370e => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:10.958216 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabda12c-4a09-4bf5-a89e-91c96baa8015 service neutron] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:10.959659 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-aabda12c-4a09-4bf5-a89e-91c96baa8015 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:10.972466 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabda12c-4a09-4bf5-a89e-91c96baa8015 service neutron] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:11.012386 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c36cbc-1b05-4eb5-af9e-6041b6bb9fab admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:11.013801 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-a1c36cbc-1b05-4eb5-af9e-6041b6bb9fab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:11.035261 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:11.071462 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964cbe31-9f77-4ccb-8894-2d3312675ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:11.072182 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964cbe31-9f77-4ccb-8894-2d3312675ff0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:11.072464 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964cbe31-9f77-4ccb-8894-2d3312675ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:11.072735 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964cbe31-9f77-4ccb-8894-2d3312675ff0 admin admin] PATH_INFO: `/v3/projects/8a1054585ddc4cc39a4f5e721b8f240c` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:11.073847 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964cbe31-9f77-4ccb-8894-2d3312675ff0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a1054585ddc4cc39a4f5e721b8f240c)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:11.075829 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964cbe31-9f77-4ccb-8894-2d3312675ff0 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:11.094073 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c36cbc-1b05-4eb5-af9e-6041b6bb9fab admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:11.098799 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8549a0-07ef-4df7-960b-7e6129da6b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:11.099462 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8549a0-07ef-4df7-960b-7e6129da6b7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:11.099741 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8549a0-07ef-4df7-960b-7e6129da6b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:11.100011 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8549a0-07ef-4df7-960b-7e6129da6b7b admin admin] PATH_INFO: `/v3/projects/adf0cc68cb1245e3a867cf6dcee4b882` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:11.101225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8549a0-07ef-4df7-960b-7e6129da6b7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=adf0cc68cb1245e3a867cf6dcee4b882)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:11.103170 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8549a0-07ef-4df7-960b-7e6129da6b7b admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:11.131066 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-964cbe31-9f77-4ccb-8894-2d3312675ff0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a1054585ddc4cc39a4f5e721b8f240c', 'request_id': 'req-964cbe31-9f77-4ccb-8894-2d3312675ff0'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:11.139143 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-964cbe31-9f77-4ccb-8894-2d3312675ff0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a1054585ddc4cc39a4f5e721b8f240c', 'request_id': 'req-964cbe31-9f77-4ccb-8894-2d3312675ff0'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:11.147301 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-964cbe31-9f77-4ccb-8894-2d3312675ff0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a1054585ddc4cc39a4f5e721b8f240c', 'request_id': 'req-964cbe31-9f77-4ccb-8894-2d3312675ff0'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:11.161478 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-964cbe31-9f77-4ccb-8894-2d3312675ff0 admin admin] The token cache is being invalidate because project 8a1054585ddc4cc39a4f5e721b8f240c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:27:11.161727 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-964cbe31-9f77-4ccb-8894-2d3312675ff0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:11.161893 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-1c8549a0-07ef-4df7-960b-7e6129da6b7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adf0cc68cb1245e3a867cf6dcee4b882', 'request_id': 'req-1c8549a0-07ef-4df7-960b-7e6129da6b7b'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:11.162380 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-964cbe31-9f77-4ccb-8894-2d3312675ff0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:11.164428 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 993/1980] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:27:10 2022] DELETE /identity/v3/projects/8a1054585ddc4cc39a4f5e721b8f240c => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:11.168469 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964cbe31-9f77-4ccb-8894-2d3312675ff0 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:11.168852 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-1c8549a0-07ef-4df7-960b-7e6129da6b7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adf0cc68cb1245e3a867cf6dcee4b882', 'request_id': 'req-1c8549a0-07ef-4df7-960b-7e6129da6b7b'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:11.170335 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-964cbe31-9f77-4ccb-8894-2d3312675ff0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:11.174912 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-1c8549a0-07ef-4df7-960b-7e6129da6b7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adf0cc68cb1245e3a867cf6dcee4b882', 'request_id': 'req-1c8549a0-07ef-4df7-960b-7e6129da6b7b'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:11.195194 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-1c8549a0-07ef-4df7-960b-7e6129da6b7b admin admin] The token cache is being invalidate because project adf0cc68cb1245e3a867cf6dcee4b882 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:27:11.195535 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-1c8549a0-07ef-4df7-960b-7e6129da6b7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:11.196170 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-1c8549a0-07ef-4df7-960b-7e6129da6b7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:11.197648 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 988/1981] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:27:11 2022] DELETE /identity/v3/projects/adf0cc68cb1245e3a867cf6dcee4b882 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:27:11.203201 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674cbe3c-0e77-4e52-aff5-fa73677bd307 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:11.203542 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674cbe3c-0e77-4e52-aff5-fa73677bd307 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:11.203825 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674cbe3c-0e77-4e52-aff5-fa73677bd307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:11.223597 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-674cbe3c-0e77-4e52-aff5-fa73677bd307 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:11.224802 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-674cbe3c-0e77-4e52-aff5-fa73677bd307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:11.238029 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964cbe31-9f77-4ccb-8894-2d3312675ff0 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:11.272185 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:11.287572 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 989/1982] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:27:11.295400 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f3657-268d-4ab8-a4af-b7e5cf8aca53 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:11.295685 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f3657-268d-4ab8-a4af-b7e5cf8aca53 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:11.295961 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f3657-268d-4ab8-a4af-b7e5cf8aca53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:11.305207 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085a1f64-00e3-4391-b674-148b1648b874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:11.305951 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a1f64-00e3-4391-b674-148b1648b874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:11.306229 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a1f64-00e3-4391-b674-148b1648b874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:11.306499 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a1f64-00e3-4391-b674-148b1648b874 admin admin] PATH_INFO: `/v3/users/c83d9c9fc7db4ba2ba1cfe9dc678b3bf` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:11.307731 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085a1f64-00e3-4391-b674-148b1648b874 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c83d9c9fc7db4ba2ba1cfe9dc678b3bf)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:11.309759 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085a1f64-00e3-4391-b674-148b1648b874 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:11.315429 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-0e6f3657-268d-4ab8-a4af-b7e5cf8aca53 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:11.316613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-0e6f3657-268d-4ab8-a4af-b7e5cf8aca53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:11.324354 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 990/1983] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:27:11.328615 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f3657-268d-4ab8-a4af-b7e5cf8aca53 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:11.338213 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f3657-268d-4ab8-a4af-b7e5cf8aca53 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:11.343762 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0630ca86-25b4-4fd5-84b0-ff525c899efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:11.344361 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630ca86-25b4-4fd5-84b0-ff525c899efe admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:11.345005 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630ca86-25b4-4fd5-84b0-ff525c899efe admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:11.345278 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630ca86-25b4-4fd5-84b0-ff525c899efe admin admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:11.346072 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0630ca86-25b4-4fd5-84b0-ff525c899efe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:27:11.346373 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0630ca86-25b4-4fd5-84b0-ff525c899efe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:11.348065 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0630ca86-25b4-4fd5-84b0-ff525c899efe admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:11.357479 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-085a1f64-00e3-4391-b674-148b1648b874 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c83d9c9fc7db4ba2ba1cfe9dc678b3bf'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:11.358450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-085a1f64-00e3-4391-b674-148b1648b874 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c83d9c9fc7db4ba2ba1cfe9dc678b3bf'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:11.359096 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-085a1f64-00e3-4391-b674-148b1648b874 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c83d9c9fc7db4ba2ba1cfe9dc678b3bf'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:11.360328 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 991/1984] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:27:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:27:11.364482 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0630ca86-25b4-4fd5-84b0-ff525c899efe admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:11.366178 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-0630ca86-25b4-4fd5-84b0-ff525c899efe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:11.372208 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-085a1f64-00e3-4391-b674-148b1648b874 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c83d9c9fc7db4ba2ba1cfe9dc678b3bf'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:11.379041 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-085a1f64-00e3-4391-b674-148b1648b874 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c83d9c9fc7db4ba2ba1cfe9dc678b3bf'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:11.379640 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0630ca86-25b4-4fd5-84b0-ff525c899efe admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:11.387450 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25b5676-22b3-420b-b34f-36dd75de575e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:11.387450 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b5676-22b3-420b-b34f-36dd75de575e admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:11.387450 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b5676-22b3-420b-b34f-36dd75de575e admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:11.387450 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b5676-22b3-420b-b34f-36dd75de575e admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:11.387864 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25b5676-22b3-420b-b34f-36dd75de575e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:11.387960 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25b5676-22b3-420b-b34f-36dd75de575e admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:11.394255 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 994/1985] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:27:11 2022] DELETE /identity/v3/users/c83d9c9fc7db4ba2ba1cfe9dc678b3bf => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:27:11.410405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:11.422339 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 992/1986] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:27:11 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:27:11.423384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085a1f64-00e3-4391-b674-148b1648b874 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:11.494699 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085a1f64-00e3-4391-b674-148b1648b874 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:11.600319 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f3f2b8-08b5-41bb-a952-6cbaee0fe5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:11.601644 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3f2b8-08b5-41bb-a952-6cbaee0fe5c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:11.601923 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3f2b8-08b5-41bb-a952-6cbaee0fe5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:11.602792 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3f2b8-08b5-41bb-a952-6cbaee0fe5c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:11.603642 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f3f2b8-08b5-41bb-a952-6cbaee0fe5c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:11.607533 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f3f2b8-08b5-41bb-a952-6cbaee0fe5c6 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:11.620458 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:11.644963 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 995/1987] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:27:11 2022] POST /identity/v3/users => generated 292 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:27:11.649498 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25b5676-22b3-420b-b34f-36dd75de575e admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:11.660803 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25b5676-22b3-420b-b34f-36dd75de575e admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:11.667346 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffd24a9-468e-464c-bf10-34f391d040b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:11.668024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd24a9-468e-464c-bf10-34f391d040b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:11.668305 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd24a9-468e-464c-bf10-34f391d040b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:11.668577 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd24a9-468e-464c-bf10-34f391d040b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:11.669395 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffd24a9-468e-464c-bf10-34f391d040b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:11.671321 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffd24a9-468e-464c-bf10-34f391d040b0 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:11.685798 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 993/1988] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:27:11.686461 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f3f2b8-08b5-41bb-a952-6cbaee0fe5c6 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:11.700900 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f3f2b8-08b5-41bb-a952-6cbaee0fe5c6 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:11.710945 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a8164c-2f19-4616-9c8a-1d216a33ff6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:11.714541 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8164c-2f19-4616-9c8a-1d216a33ff6c admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:11.715074 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8164c-2f19-4616-9c8a-1d216a33ff6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:11.715513 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8164c-2f19-4616-9c8a-1d216a33ff6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:11.726453 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffd24a9-468e-464c-bf10-34f391d040b0 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:11.727548 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a8164c-2f19-4616-9c8a-1d216a33ff6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:11.728486 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-cffd24a9-468e-464c-bf10-34f391d040b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:11.730879 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a8164c-2f19-4616-9c8a-1d216a33ff6c admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:11.746745 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffd24a9-468e-464c-bf10-34f391d040b0 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:11.753068 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f7d5d3-9618-40b2-ad27-e9a3f6802ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:11.753767 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7d5d3-9618-40b2-ad27-e9a3f6802ef6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:11.754189 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7d5d3-9618-40b2-ad27-e9a3f6802ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:11.754530 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7d5d3-9618-40b2-ad27-e9a3f6802ef6 admin admin] PATH_INFO: `/v3/projects/3c01fb2231f1458db5bc419f3dd66629` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:11.755970 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f7d5d3-9618-40b2-ad27-e9a3f6802ef6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c01fb2231f1458db5bc419f3dd66629)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:11.757986 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f7d5d3-9618-40b2-ad27-e9a3f6802ef6 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:11.771930 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 996/1989] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 14:27:11.775573 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a8164c-2f19-4616-9c8a-1d216a33ff6c admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:11.787876 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a8164c-2f19-4616-9c8a-1d216a33ff6c admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:11.793832 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898f237-bdac-4832-9adc-102df29a2a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:11.794498 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898f237-bdac-4832-9adc-102df29a2a2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:11.794909 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898f237-bdac-4832-9adc-102df29a2a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:11.795071 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898f237-bdac-4832-9adc-102df29a2a2d admin admin] PATH_INFO: `/v3/projects/821bfb03944a446681061d24e1a5d12e/users/5969c0c9fc9c4af188be88cae9fddf8a/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:11.810891 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6898f237-bdac-4832-9adc-102df29a2a2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=821bfb03944a446681061d24e1a5d12e, user_id=5969c0c9fc9c4af188be88cae9fddf8a, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:11.813560 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6898f237-bdac-4832-9adc-102df29a2a2d admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:11.816761 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-53f7d5d3-9618-40b2-ad27-e9a3f6802ef6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c01fb2231f1458db5bc419f3dd66629', 'request_id': 'req-53f7d5d3-9618-40b2-ad27-e9a3f6802ef6'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:11.825018 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-53f7d5d3-9618-40b2-ad27-e9a3f6802ef6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c01fb2231f1458db5bc419f3dd66629', 'request_id': 'req-53f7d5d3-9618-40b2-ad27-e9a3f6802ef6'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:11.830368 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 997/1990] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:27:11 2022] PUT /identity/v3/projects/821bfb03944a446681061d24e1a5d12e/users/5969c0c9fc9c4af188be88cae9fddf8a/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:27:11.833138 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-53f7d5d3-9618-40b2-ad27-e9a3f6802ef6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c01fb2231f1458db5bc419f3dd66629', 'request_id': 'req-53f7d5d3-9618-40b2-ad27-e9a3f6802ef6'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:11.842151 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd5cdb-d8f1-49f5-ad60-25c3c141687e None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:11.842509 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd5cdb-d8f1-49f5-ad60-25c3c141687e None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:11.842876 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd5cdb-d8f1-49f5-ad60-25c3c141687e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:11.846052 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-53f7d5d3-9618-40b2-ad27-e9a3f6802ef6 admin admin] The token cache is being invalidate because project 3c01fb2231f1458db5bc419f3dd66629 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:27:11.846917 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-53f7d5d3-9618-40b2-ad27-e9a3f6802ef6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:11.847022 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-53f7d5d3-9618-40b2-ad27-e9a3f6802ef6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:11.851140 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 994/1991] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:27:11 2022] DELETE /identity/v3/projects/3c01fb2231f1458db5bc419f3dd66629 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:27:11.853909 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b432d-3e0c-49c6-abd0-cce92c9ab08b None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:11.854193 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b432d-3e0c-49c6-abd0-cce92c9ab08b None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:11.854471 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b432d-3e0c-49c6-abd0-cce92c9ab08b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:11.873045 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-83bd5cdb-d8f1-49f5-ad60-25c3c141687e None None] MFA Rules not processed for user `5969c0c9fc9c4af188be88cae9fddf8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:11.873627 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-004b432d-3e0c-49c6-abd0-cce92c9ab08b None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:11.874246 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-83bd5cdb-d8f1-49f5-ad60-25c3c141687e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:11.874770 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-004b432d-3e0c-49c6-abd0-cce92c9ab08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:11.934839 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 995/1992] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:27:11.939589 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 998/1993] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:27:11.942307 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc078ae-34c5-42b2-85fa-bf102a45207e None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:11.942647 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc078ae-34c5-42b2-85fa-bf102a45207e None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:11.942960 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc078ae-34c5-42b2-85fa-bf102a45207e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:11.951003 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd5cdb-d8f1-49f5-ad60-25c3c141687e None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:11.952363 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-83bd5cdb-d8f1-49f5-ad60-25c3c141687e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:11.965121 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-1cc078ae-34c5-42b2-85fa-bf102a45207e None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:11.966315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-1cc078ae-34c5-42b2-85fa-bf102a45207e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:11.979080 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 996/1994] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:27:11.980269 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc078ae-34c5-42b2-85fa-bf102a45207e None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:11.989767 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc078ae-34c5-42b2-85fa-bf102a45207e None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:11.994555 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fec97f-93b8-4dae-9cd9-b0416af566b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:11.995196 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fec97f-93b8-4dae-9cd9-b0416af566b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:11.995477 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fec97f-93b8-4dae-9cd9-b0416af566b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:11.995747 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fec97f-93b8-4dae-9cd9-b0416af566b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:11.996401 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fec97f-93b8-4dae-9cd9-b0416af566b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:27:11.996697 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fec97f-93b8-4dae-9cd9-b0416af566b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:11.998294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fec97f-93b8-4dae-9cd9-b0416af566b2 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:12.023244 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 997/1995] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:27:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:27:12.024517 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fec97f-93b8-4dae-9cd9-b0416af566b2 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:12.026441 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd5cdb-d8f1-49f5-ad60-25c3c141687e None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:12.030288 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616cddf-d1d0-4ee6-a347-1379c7cdaa32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:12.031056 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616cddf-d1d0-4ee6-a347-1379c7cdaa32 service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:12.031192 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616cddf-d1d0-4ee6-a347-1379c7cdaa32 service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:12.031457 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616cddf-d1d0-4ee6-a347-1379c7cdaa32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:12.034854 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fec97f-93b8-4dae-9cd9-b0416af566b2 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:12.039874 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52caaa7-5de5-40c0-81d6-52ecb251fed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:12.041684 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6616cddf-d1d0-4ee6-a347-1379c7cdaa32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:12.041801 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52caaa7-5de5-40c0-81d6-52ecb251fed8 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:12.041801 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52caaa7-5de5-40c0-81d6-52ecb251fed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:12.041801 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52caaa7-5de5-40c0-81d6-52ecb251fed8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:12.041801 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52caaa7-5de5-40c0-81d6-52ecb251fed8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:12.043739 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52caaa7-5de5-40c0-81d6-52ecb251fed8 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:12.050333 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6616cddf-d1d0-4ee6-a347-1379c7cdaa32 service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:12.060616 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 999/1996] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:27:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 14:27:12.078134 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 998/1997] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:27:12 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:27:12.082419 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616cddf-d1d0-4ee6-a347-1379c7cdaa32 service cinder] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:12.089694 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52caaa7-5de5-40c0-81d6-52ecb251fed8 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:12.093167 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-a52caaa7-5de5-40c0-81d6-52ecb251fed8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:12.100212 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616cddf-d1d0-4ee6-a347-1379c7cdaa32 service cinder] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:12.135855 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:12.161072 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:12.188783 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52caaa7-5de5-40c0-81d6-52ecb251fed8 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:12.192471 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae8974-a54d-4ea8-af3a-8b843fe3a82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:12.192471 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae8974-a54d-4ea8-af3a-8b843fe3a82e admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:12.192709 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae8974-a54d-4ea8-af3a-8b843fe3a82e admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:12.192910 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae8974-a54d-4ea8-af3a-8b843fe3a82e admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:12.193567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cae8974-a54d-4ea8-af3a-8b843fe3a82e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:12.196077 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cae8974-a54d-4ea8-af3a-8b843fe3a82e admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:12.201996 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c4727-1590-4ecd-86a8-926a9d98c121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:12.202910 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c4727-1590-4ecd-86a8-926a9d98c121 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:12.203065 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c4727-1590-4ecd-86a8-926a9d98c121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:12.203291 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c4727-1590-4ecd-86a8-926a9d98c121 admin admin] PATH_INFO: `/v3/users/5969c0c9fc9c4af188be88cae9fddf8a` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:12.204760 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c4727-1590-4ecd-86a8-926a9d98c121 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5969c0c9fc9c4af188be88cae9fddf8a)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:12.207144 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c4727-1590-4ecd-86a8-926a9d98c121 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:12.229083 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1000/1998] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:27:12 2022] POST /identity/v3/users => generated 290 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:27:12.230480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae8974-a54d-4ea8-af3a-8b843fe3a82e admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:12.248086 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae8974-a54d-4ea8-af3a-8b843fe3a82e admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:12.257680 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a263d6-36db-4b20-b032-ed60cf3c70ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:12.258476 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a263d6-36db-4b20-b032-ed60cf3c70ad admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:12.258838 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a263d6-36db-4b20-b032-ed60cf3c70ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:12.259189 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a263d6-36db-4b20-b032-ed60cf3c70ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:12.259937 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a263d6-36db-4b20-b032-ed60cf3c70ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:12.261650 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a263d6-36db-4b20-b032-ed60cf3c70ad admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:12.265692 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-5c9c4727-1590-4ecd-86a8-926a9d98c121 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5969c0c9fc9c4af188be88cae9fddf8a'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:12.266564 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-5c9c4727-1590-4ecd-86a8-926a9d98c121 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5969c0c9fc9c4af188be88cae9fddf8a'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:12.272239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1001/1999] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:27:12.276569 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a263d6-36db-4b20-b032-ed60cf3c70ad admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:12.277341 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-5c9c4727-1590-4ecd-86a8-926a9d98c121 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5969c0c9fc9c4af188be88cae9fddf8a'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:12.278059 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-5c9c4727-1590-4ecd-86a8-926a9d98c121 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5969c0c9fc9c4af188be88cae9fddf8a'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:12.278580 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-95a263d6-36db-4b20-b032-ed60cf3c70ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:12.290470 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-5c9c4727-1590-4ecd-86a8-926a9d98c121 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5969c0c9fc9c4af188be88cae9fddf8a'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:12.291947 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a263d6-36db-4b20-b032-ed60cf3c70ad admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:12.297095 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd1c135-7630-45e4-95c8-b959600fa8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:12.297684 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1c135-7630-45e4-95c8-b959600fa8a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:12.298108 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1c135-7630-45e4-95c8-b959600fa8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:12.298373 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1c135-7630-45e4-95c8-b959600fa8a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:12.299069 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd1c135-7630-45e4-95c8-b959600fa8a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:12.300617 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd1c135-7630-45e4-95c8-b959600fa8a8 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:12.304242 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 999/2000] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:27:12 2022] DELETE /identity/v3/users/5969c0c9fc9c4af188be88cae9fddf8a => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:12.308379 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3cc8c8-91ae-4602-b4ff-35bc305a7c4a None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:12.308659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3cc8c8-91ae-4602-b4ff-35bc305a7c4a None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:12.308943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3cc8c8-91ae-4602-b4ff-35bc305a7c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:12.312815 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1002/2001] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 14:27:12.317441 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd1c135-7630-45e4-95c8-b959600fa8a8 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:12.326874 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd1c135-7630-45e4-95c8-b959600fa8a8 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:12.328221 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-0f3cc8c8-91ae-4602-b4ff-35bc305a7c4a None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:12.329405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-0f3cc8c8-91ae-4602-b4ff-35bc305a7c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:12.383166 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb01ea1-2a78-423d-8964-f7839a7ffe37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:12.383933 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb01ea1-2a78-423d-8964-f7839a7ffe37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:12.384284 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb01ea1-2a78-423d-8964-f7839a7ffe37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:12.384489 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb01ea1-2a78-423d-8964-f7839a7ffe37 admin admin] PATH_INFO: `/v3/projects/46aa71a8a6414987bec0c395b23cc1bd/users/ad2f391e9d6448b09298892f281af639/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:12.389178 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1000/2002] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:27:12.398888 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3cc8c8-91ae-4602-b4ff-35bc305a7c4a None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:12.400265 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-0f3cc8c8-91ae-4602-b4ff-35bc305a7c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:12.401721 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb01ea1-2a78-423d-8964-f7839a7ffe37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46aa71a8a6414987bec0c395b23cc1bd, user_id=ad2f391e9d6448b09298892f281af639, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:12.406654 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb01ea1-2a78-423d-8964-f7839a7ffe37 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:12.425253 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1003/2003] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:27:12 2022] PUT /identity/v3/projects/46aa71a8a6414987bec0c395b23cc1bd/users/ad2f391e9d6448b09298892f281af639/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:27:12.436809 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de339801-df35-4a0a-80f4-298ca0a80606 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:12.437092 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de339801-df35-4a0a-80f4-298ca0a80606 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:12.437373 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de339801-df35-4a0a-80f4-298ca0a80606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:12.455876 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:12.461244 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3cc8c8-91ae-4602-b4ff-35bc305a7c4a None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:12.466680 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-de339801-df35-4a0a-80f4-298ca0a80606 None None] MFA Rules not processed for user `ad2f391e9d6448b09298892f281af639`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:12.468088 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-de339801-df35-4a0a-80f4-298ca0a80606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:12.511412 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41867e08-ed13-491e-80a1-cdb6743d0652 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:12.512024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41867e08-ed13-491e-80a1-cdb6743d0652 service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:12.512299 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41867e08-ed13-491e-80a1-cdb6743d0652 service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:12.512568 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41867e08-ed13-491e-80a1-cdb6743d0652 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:12.535200 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1004/2004] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:27:12.539727 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de339801-df35-4a0a-80f4-298ca0a80606 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:12.570230 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:12.574922 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41867e08-ed13-491e-80a1-cdb6743d0652 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:12.576536 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41867e08-ed13-491e-80a1-cdb6743d0652 service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:12.588287 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1001/2005] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:27:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:27:12.605044 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de339801-df35-4a0a-80f4-298ca0a80606 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:12.609722 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961a79bc-b94a-44eb-93d1-a1a299e28ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:12.610343 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a79bc-b94a-44eb-93d1-a1a299e28ff2 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:12.610703 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a79bc-b94a-44eb-93d1-a1a299e28ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:12.610902 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a79bc-b94a-44eb-93d1-a1a299e28ff2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:12.611686 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961a79bc-b94a-44eb-93d1-a1a299e28ff2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:12.613967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961a79bc-b94a-44eb-93d1-a1a299e28ff2 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:12.652666 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1005/2006] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:27:12 2022] POST /identity/v3/projects => generated 379 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:27:12.654148 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41867e08-ed13-491e-80a1-cdb6743d0652 service neutron] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:12.665836 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41867e08-ed13-491e-80a1-cdb6743d0652 service neutron] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:12.723629 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3199a49-4c89-49bb-ad00-133426a5a781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:12.724158 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3199a49-4c89-49bb-ad00-133426a5a781 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:12.724435 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3199a49-4c89-49bb-ad00-133426a5a781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:12.724704 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3199a49-4c89-49bb-ad00-133426a5a781 admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:12.725330 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3199a49-4c89-49bb-ad00-133426a5a781 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:12.727815 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3199a49-4c89-49bb-ad00-133426a5a781 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:12.754875 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1002/2007] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:27:12 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:27:12.758842 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961a79bc-b94a-44eb-93d1-a1a299e28ff2 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:12.769496 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961a79bc-b94a-44eb-93d1-a1a299e28ff2 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:12.773913 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efae3ee-f270-482a-99a8-1f086fbfe9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:12.774524 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efae3ee-f270-482a-99a8-1f086fbfe9df admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:12.774805 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efae3ee-f270-482a-99a8-1f086fbfe9df admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:12.775115 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efae3ee-f270-482a-99a8-1f086fbfe9df admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:12.775757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efae3ee-f270-482a-99a8-1f086fbfe9df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:12.777337 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efae3ee-f270-482a-99a8-1f086fbfe9df admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:12.792233 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1006/2008] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:27:12.796059 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3199a49-4c89-49bb-ad00-133426a5a781 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:12.806795 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3199a49-4c89-49bb-ad00-133426a5a781 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:12.812277 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ad5d3-54fb-48f5-9d25-83c90c0dcc1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:12.813542 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ad5d3-54fb-48f5-9d25-83c90c0dcc1f admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:12.813826 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ad5d3-54fb-48f5-9d25-83c90c0dcc1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:12.814105 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ad5d3-54fb-48f5-9d25-83c90c0dcc1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:12.814757 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ad5d3-54fb-48f5-9d25-83c90c0dcc1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:12.818902 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ad5d3-54fb-48f5-9d25-83c90c0dcc1f admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:12.828244 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1003/2009] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 14:27:12.832418 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efae3ee-f270-482a-99a8-1f086fbfe9df admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:12.842772 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efae3ee-f270-482a-99a8-1f086fbfe9df admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:12.847462 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db04decd-1c19-4aff-a734-4a19c2026d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:12.848117 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db04decd-1c19-4aff-a734-4a19c2026d53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:12.848392 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db04decd-1c19-4aff-a734-4a19c2026d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:12.848663 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db04decd-1c19-4aff-a734-4a19c2026d53 admin admin] PATH_INFO: `/v3/projects/079998319ccc4f73a169e24dc5afadde/users/080a54cdffc24f7fade20dfcb7b2bffe/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:12.864482 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db04decd-1c19-4aff-a734-4a19c2026d53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=079998319ccc4f73a169e24dc5afadde, user_id=080a54cdffc24f7fade20dfcb7b2bffe, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:12.867738 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db04decd-1c19-4aff-a734-4a19c2026d53 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:12.881094 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1007/2010] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:27:12 2022] PUT /identity/v3/projects/079998319ccc4f73a169e24dc5afadde/users/080a54cdffc24f7fade20dfcb7b2bffe/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:27:12.891669 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89198986-b5a7-4948-851f-0a988a62a1e7 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:12.891961 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89198986-b5a7-4948-851f-0a988a62a1e7 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:12.892239 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89198986-b5a7-4948-851f-0a988a62a1e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:12.926248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-89198986-b5a7-4948-851f-0a988a62a1e7 None None] MFA Rules not processed for user `080a54cdffc24f7fade20dfcb7b2bffe`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:12.927524 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-89198986-b5a7-4948-851f-0a988a62a1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:12.944801 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db04decd-1c19-4aff-a734-4a19c2026d53 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:12.946689 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-db04decd-1c19-4aff-a734-4a19c2026d53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:12.955942 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:13.013429 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1004/2011] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:27:13.018036 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89198986-b5a7-4948-851f-0a988a62a1e7 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:13.023140 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db04decd-1c19-4aff-a734-4a19c2026d53 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:13.028276 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51c5e4-32e4-4d3b-ab64-e0b9333e4a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:13.029487 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51c5e4-32e4-4d3b-ab64-e0b9333e4a6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:13.029487 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51c5e4-32e4-4d3b-ab64-e0b9333e4a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:13.029663 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89198986-b5a7-4948-851f-0a988a62a1e7 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:13.029871 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51c5e4-32e4-4d3b-ab64-e0b9333e4a6c admin admin] PATH_INFO: `/v3/projects/821bfb03944a446681061d24e1a5d12e` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:13.031209 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec51c5e4-32e4-4d3b-ab64-e0b9333e4a6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=821bfb03944a446681061d24e1a5d12e)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:13.033747 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec51c5e4-32e4-4d3b-ab64-e0b9333e4a6c admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:13.034773 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df13f96-abda-44ec-bb27-01b999d28c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:13.035478 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df13f96-abda-44ec-bb27-01b999d28c6f admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:13.035831 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df13f96-abda-44ec-bb27-01b999d28c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:13.036169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df13f96-abda-44ec-bb27-01b999d28c6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:13.036893 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df13f96-abda-44ec-bb27-01b999d28c6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:13.039601 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df13f96-abda-44ec-bb27-01b999d28c6f admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:13.070872 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1005/2012] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:27:13 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:27:13.080044 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df13f96-abda-44ec-bb27-01b999d28c6f admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:13.091110 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-ec51c5e4-32e4-4d3b-ab64-e0b9333e4a6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '821bfb03944a446681061d24e1a5d12e', 'request_id': 'req-ec51c5e4-32e4-4d3b-ab64-e0b9333e4a6c'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:13.097321 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-ec51c5e4-32e4-4d3b-ab64-e0b9333e4a6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '821bfb03944a446681061d24e1a5d12e', 'request_id': 'req-ec51c5e4-32e4-4d3b-ab64-e0b9333e4a6c'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:13.103143 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-ec51c5e4-32e4-4d3b-ab64-e0b9333e4a6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '821bfb03944a446681061d24e1a5d12e', 'request_id': 'req-ec51c5e4-32e4-4d3b-ab64-e0b9333e4a6c'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:13.116652 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-ec51c5e4-32e4-4d3b-ab64-e0b9333e4a6c admin admin] The token cache is being invalidate because project 821bfb03944a446681061d24e1a5d12e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:27:13.116951 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-ec51c5e4-32e4-4d3b-ab64-e0b9333e4a6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:13.118407 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-ec51c5e4-32e4-4d3b-ab64-e0b9333e4a6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:13.120021 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1008/2013] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:27:12 2022] DELETE /identity/v3/projects/821bfb03944a446681061d24e1a5d12e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:27:13.124685 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361b4ef-bd8f-4922-9e60-4d031b410c89 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:13.124972 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361b4ef-bd8f-4922-9e60-4d031b410c89 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:13.125249 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361b4ef-bd8f-4922-9e60-4d031b410c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:13.145253 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df13f96-abda-44ec-bb27-01b999d28c6f admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:13.145450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-f361b4ef-bd8f-4922-9e60-4d031b410c89 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:13.146615 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-f361b4ef-bd8f-4922-9e60-4d031b410c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:13.150872 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:13.204705 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ac430-9d3b-4f9b-b05c-f5685f710925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:13.205333 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ac430-9d3b-4f9b-b05c-f5685f710925 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:13.205617 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ac430-9d3b-4f9b-b05c-f5685f710925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:13.205894 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ac430-9d3b-4f9b-b05c-f5685f710925 admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:13.206543 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ac430-9d3b-4f9b-b05c-f5685f710925 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:13.208642 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ac430-9d3b-4f9b-b05c-f5685f710925 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:13.212475 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1009/2014] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:27:13.222013 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a9686-8046-428d-9d00-095079008b94 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:13.222300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a9686-8046-428d-9d00-095079008b94 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:13.222952 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a9686-8046-428d-9d00-095079008b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:13.235491 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1006/2015] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:27:13 2022] POST /identity/v3/users => generated 296 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:27:13.239760 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ac430-9d3b-4f9b-b05c-f5685f710925 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:13.243285 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-ef5ac430-9d3b-4f9b-b05c-f5685f710925 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:13.244767 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-314a9686-8046-428d-9d00-095079008b94 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:13.246225 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-314a9686-8046-428d-9d00-095079008b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:13.255770 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ac430-9d3b-4f9b-b05c-f5685f710925 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:13.258597 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1010/2016] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:27:13.259731 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a9686-8046-428d-9d00-095079008b94 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:13.262012 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46911a25-8c47-4b92-858c-fe5761b9d659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:13.262706 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46911a25-8c47-4b92-858c-fe5761b9d659 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:13.263123 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46911a25-8c47-4b92-858c-fe5761b9d659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:13.263499 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46911a25-8c47-4b92-858c-fe5761b9d659 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:13.264252 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46911a25-8c47-4b92-858c-fe5761b9d659 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:13.265946 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46911a25-8c47-4b92-858c-fe5761b9d659 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:13.273811 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a9686-8046-428d-9d00-095079008b94 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:13.280563 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1007/2017] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:27:13.284338 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46911a25-8c47-4b92-858c-fe5761b9d659 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:13.291693 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae269946-e044-419f-a64d-3f90bbf05988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:13.292440 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae269946-e044-419f-a64d-3f90bbf05988 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:13.292929 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae269946-e044-419f-a64d-3f90bbf05988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:13.293412 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae269946-e044-419f-a64d-3f90bbf05988 admin admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:13.294309 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae269946-e044-419f-a64d-3f90bbf05988 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:27:13.295827 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae269946-e044-419f-a64d-3f90bbf05988 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:13.297660 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae269946-e044-419f-a64d-3f90bbf05988 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:13.300287 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46911a25-8c47-4b92-858c-fe5761b9d659 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:13.310757 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce912240-1d91-4d9b-855a-311f8a292007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:13.311609 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce912240-1d91-4d9b-855a-311f8a292007 admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:13.311989 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce912240-1d91-4d9b-855a-311f8a292007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:13.312365 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce912240-1d91-4d9b-855a-311f8a292007 admin admin] PATH_INFO: `/v3/projects/4684c7f3a5b947b88586b1250bb37885/users/052c60814af948ce913db19f1756ef5a/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:13.318435 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1011/2018] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:27:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:27:13.326243 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae269946-e044-419f-a64d-3f90bbf05988 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:13.336987 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce912240-1d91-4d9b-855a-311f8a292007 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4684c7f3a5b947b88586b1250bb37885, user_id=052c60814af948ce913db19f1756ef5a, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:13.337205 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae269946-e044-419f-a64d-3f90bbf05988 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:13.340265 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce912240-1d91-4d9b-855a-311f8a292007 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:13.343361 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746fd691-1624-4aa3-b1d6-2fbd136ca6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:13.343987 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746fd691-1624-4aa3-b1d6-2fbd136ca6d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:13.344265 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746fd691-1624-4aa3-b1d6-2fbd136ca6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:13.344535 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746fd691-1624-4aa3-b1d6-2fbd136ca6d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:13.347299 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746fd691-1624-4aa3-b1d6-2fbd136ca6d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:13.349447 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746fd691-1624-4aa3-b1d6-2fbd136ca6d5 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:13.360559 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1008/2019] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:27:13 2022] PUT /identity/v3/projects/4684c7f3a5b947b88586b1250bb37885/users/052c60814af948ce913db19f1756ef5a/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:13.361481 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce912240-1d91-4d9b-855a-311f8a292007 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:13.368331 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:13.386878 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1012/2020] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:27:13 2022] POST /identity/v3/projects => generated 375 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:27:13.390856 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746fd691-1624-4aa3-b1d6-2fbd136ca6d5 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:13.442628 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce912240-1d91-4d9b-855a-311f8a292007 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:13.469590 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746fd691-1624-4aa3-b1d6-2fbd136ca6d5 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:13.477264 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84cff51-d8b6-47a8-8b4f-2fcf229bed7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:13.477896 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84cff51-d8b6-47a8-8b4f-2fcf229bed7b admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:13.478218 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84cff51-d8b6-47a8-8b4f-2fcf229bed7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:13.478500 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84cff51-d8b6-47a8-8b4f-2fcf229bed7b admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:13.479169 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84cff51-d8b6-47a8-8b4f-2fcf229bed7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:13.481116 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84cff51-d8b6-47a8-8b4f-2fcf229bed7b admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:13.503657 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1013/2021] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:27:13 2022] POST /identity/v3/users => generated 288 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:27:13.508182 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84cff51-d8b6-47a8-8b4f-2fcf229bed7b admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:13.510323 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771bddf7-2a7a-4634-b2be-8f89d6c79a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:13.510965 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771bddf7-2a7a-4634-b2be-8f89d6c79a00 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:13.511242 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771bddf7-2a7a-4634-b2be-8f89d6c79a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:13.511511 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771bddf7-2a7a-4634-b2be-8f89d6c79a00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:13.512178 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771bddf7-2a7a-4634-b2be-8f89d6c79a00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:13.513799 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771bddf7-2a7a-4634-b2be-8f89d6c79a00 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:13.517941 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84cff51-d8b6-47a8-8b4f-2fcf229bed7b admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:13.522750 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598aabbb-bdc6-484f-bce1-3b736d667f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:13.523404 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598aabbb-bdc6-484f-bce1-3b736d667f86 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:13.523687 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598aabbb-bdc6-484f-bce1-3b736d667f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:13.523807 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1009/2022] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:13 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:27:13.523961 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598aabbb-bdc6-484f-bce1-3b736d667f86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:13.524599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598aabbb-bdc6-484f-bce1-3b736d667f86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:13.526166 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598aabbb-bdc6-484f-bce1-3b736d667f86 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:13.527649 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771bddf7-2a7a-4634-b2be-8f89d6c79a00 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:13.539824 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1014/2023] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 14:27:13.539999 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771bddf7-2a7a-4634-b2be-8f89d6c79a00 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:13.541190 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598aabbb-bdc6-484f-bce1-3b736d667f86 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:13.542530 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550a97f-72c0-40a3-9802-9a4479e64e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:13.543189 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550a97f-72c0-40a3-9802-9a4479e64e18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:13.543466 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550a97f-72c0-40a3-9802-9a4479e64e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:13.543736 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550a97f-72c0-40a3-9802-9a4479e64e18 admin admin] PATH_INFO: `/v3/projects/4684c7f3a5b947b88586b1250bb37885/users/052c60814af948ce913db19f1756ef5a/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:13.545595 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0550a97f-72c0-40a3-9802-9a4479e64e18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4684c7f3a5b947b88586b1250bb37885, user_id=052c60814af948ce913db19f1756ef5a, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:13.548234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0550a97f-72c0-40a3-9802-9a4479e64e18 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:13.550962 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598aabbb-bdc6-484f-bce1-3b736d667f86 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:13.556229 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d82ef-ed9c-4134-82dc-2b60d06003ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:13.556891 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d82ef-ed9c-4134-82dc-2b60d06003ed admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:13.557180 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d82ef-ed9c-4134-82dc-2b60d06003ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:13.557458 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d82ef-ed9c-4134-82dc-2b60d06003ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:13.558085 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d82ef-ed9c-4134-82dc-2b60d06003ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:13.558833 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1010/2024] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:27:13 2022] PUT /identity/v3/projects/4684c7f3a5b947b88586b1250bb37885/users/052c60814af948ce913db19f1756ef5a/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:27:13.559677 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d82ef-ed9c-4134-82dc-2b60d06003ed admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:13.568811 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f3990-8b8f-49a6-bd64-27b7dfa9c0cd None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:13.569097 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f3990-8b8f-49a6-bd64-27b7dfa9c0cd None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:13.569379 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f3990-8b8f-49a6-bd64-27b7dfa9c0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:13.573320 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1015/2025] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 14:27:13.574093 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d82ef-ed9c-4134-82dc-2b60d06003ed admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:13.583668 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d82ef-ed9c-4134-82dc-2b60d06003ed admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:13.598027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-8c8f3990-8b8f-49a6-bd64-27b7dfa9c0cd None None] MFA Rules not processed for user `052c60814af948ce913db19f1756ef5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:13.600037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-8c8f3990-8b8f-49a6-bd64-27b7dfa9c0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:13.618737 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:13.651283 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918c04fe-c26f-4f8a-8fde-f101a11f47b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:13.651961 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c04fe-c26f-4f8a-8fde-f101a11f47b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:13.652260 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c04fe-c26f-4f8a-8fde-f101a11f47b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:13.652541 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c04fe-c26f-4f8a-8fde-f101a11f47b6 admin admin] PATH_INFO: `/v3/projects/960b936a0f964349b1641942bbd13848/users/3a2917017d544048921c4c69ed018af0/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:13.668109 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918c04fe-c26f-4f8a-8fde-f101a11f47b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=960b936a0f964349b1641942bbd13848, user_id=3a2917017d544048921c4c69ed018af0, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:13.670766 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918c04fe-c26f-4f8a-8fde-f101a11f47b6 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:13.677237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1011/2026] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:13 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:27:13.690479 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1016/2027] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:27:13 2022] PUT /identity/v3/projects/960b936a0f964349b1641942bbd13848/users/3a2917017d544048921c4c69ed018af0/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:27:13.691179 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f3990-8b8f-49a6-bd64-27b7dfa9c0cd None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:13.692569 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-8c8f3990-8b8f-49a6-bd64-27b7dfa9c0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:13.700609 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5affd-90b0-47da-8768-be1bfeb0c15f None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:13.700966 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5affd-90b0-47da-8768-be1bfeb0c15f None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:13.701307 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5affd-90b0-47da-8768-be1bfeb0c15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:13.712709 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:13.729237 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-51f5affd-90b0-47da-8768-be1bfeb0c15f None None] MFA Rules not processed for user `3a2917017d544048921c4c69ed018af0`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:13.730417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-51f5affd-90b0-47da-8768-be1bfeb0c15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:13.751742 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f3990-8b8f-49a6-bd64-27b7dfa9c0cd None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:13.797410 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1017/2028] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:27:13.801633 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f5affd-90b0-47da-8768-be1bfeb0c15f None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:13.811595 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f36d0-ad2e-486f-9850-107ba4ba6d6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:13.812670 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f36d0-ad2e-486f-9850-107ba4ba6d6f service cinder] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:13.813600 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f36d0-ad2e-486f-9850-107ba4ba6d6f service cinder] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:13.813600 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f36d0-ad2e-486f-9850-107ba4ba6d6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:13.877769 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f5affd-90b0-47da-8768-be1bfeb0c15f None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:13.881932 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7f36d0-ad2e-486f-9850-107ba4ba6d6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:13.882494 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc33ba97-3c97-45d6-b3dc-9462b74fcbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:13.883160 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33ba97-3c97-45d6-b3dc-9462b74fcbf2 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:13.883445 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33ba97-3c97-45d6-b3dc-9462b74fcbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:13.883552 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7f36d0-ad2e-486f-9850-107ba4ba6d6f service cinder] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:13.883721 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33ba97-3c97-45d6-b3dc-9462b74fcbf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:13.884353 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc33ba97-3c97-45d6-b3dc-9462b74fcbf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:13.886238 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc33ba97-3c97-45d6-b3dc-9462b74fcbf2 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:13.893078 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1012/2029] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:27:13 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 14:27:13.920138 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1018/2030] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:27:13 2022] POST /identity/v3/projects => generated 375 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:27:13.921983 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f36d0-ad2e-486f-9850-107ba4ba6d6f service cinder] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:13.938052 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f36d0-ad2e-486f-9850-107ba4ba6d6f service cinder] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:13.940683 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc33ba97-3c97-45d6-b3dc-9462b74fcbf2 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:13.942404 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-cc33ba97-3c97-45d6-b3dc-9462b74fcbf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:13.978132 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:13.999093 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65154a-7bb6-43f1-aa51-90dae03d3ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:14.000028 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65154a-7bb6-43f1-aa51-90dae03d3ffb admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:14.000308 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65154a-7bb6-43f1-aa51-90dae03d3ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:14.000582 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65154a-7bb6-43f1-aa51-90dae03d3ffb admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:14.001363 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb65154a-7bb6-43f1-aa51-90dae03d3ffb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:14.001807 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc33ba97-3c97-45d6-b3dc-9462b74fcbf2 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:14.003256 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb65154a-7bb6-43f1-aa51-90dae03d3ffb admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:14.006945 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b827f3a0-c702-4ae1-8ca9-970442bbd9fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:14.007555 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827f3a0-c702-4ae1-8ca9-970442bbd9fc service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:14.007891 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827f3a0-c702-4ae1-8ca9-970442bbd9fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:14.008167 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827f3a0-c702-4ae1-8ca9-970442bbd9fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:14.011221 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-b827f3a0-c702-4ae1-8ca9-970442bbd9fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:14.035072 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1013/2031] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:27:13 2022] POST /identity/v3/users => generated 288 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:27:14.035977 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65154a-7bb6-43f1-aa51-90dae03d3ffb admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:14.048443 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65154a-7bb6-43f1-aa51-90dae03d3ffb admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:14.054334 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c130e20-de6f-4322-8025-69cd3172b875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:14.055001 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c130e20-de6f-4322-8025-69cd3172b875 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:14.055288 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c130e20-de6f-4322-8025-69cd3172b875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:14.055555 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c130e20-de6f-4322-8025-69cd3172b875 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:14.056204 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c130e20-de6f-4322-8025-69cd3172b875 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:14.057798 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c130e20-de6f-4322-8025-69cd3172b875 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:14.071316 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1014/2032] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:27:14.075192 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c130e20-de6f-4322-8025-69cd3172b875 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:14.080369 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:14.094997 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c130e20-de6f-4322-8025-69cd3172b875 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:14.095177 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b827f3a0-c702-4ae1-8ca9-970442bbd9fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:14.095177 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b827f3a0-c702-4ae1-8ca9-970442bbd9fc service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:14.095347 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eb5e24-02e4-447f-978c-59631e9ccea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:14.095347 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb5e24-02e4-447f-978c-59631e9ccea6 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:14.095347 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb5e24-02e4-447f-978c-59631e9ccea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:14.095566 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb5e24-02e4-447f-978c-59631e9ccea6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:14.096229 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eb5e24-02e4-447f-978c-59631e9ccea6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:14.097850 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eb5e24-02e4-447f-978c-59631e9ccea6 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:14.108474 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1019/2033] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:27:13 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 14:27:14.120558 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1015/2034] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:14 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:27:14.124838 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b827f3a0-c702-4ae1-8ca9-970442bbd9fc service cinder] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:14.141674 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b827f3a0-c702-4ae1-8ca9-970442bbd9fc service cinder] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:14.146647 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195149a-34d1-4aef-8ca0-ece995ea3538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:14.147915 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195149a-34d1-4aef-8ca0-ece995ea3538 admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:14.148245 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195149a-34d1-4aef-8ca0-ece995ea3538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:14.148522 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195149a-34d1-4aef-8ca0-ece995ea3538 admin admin] PATH_INFO: `/v3/projects/da29f394f9a3424cabfb2344e894e5e5/users/10b17dfb662e42d288036d00461eca57/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:14.168754 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9195149a-34d1-4aef-8ca0-ece995ea3538 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da29f394f9a3424cabfb2344e894e5e5, user_id=10b17dfb662e42d288036d00461eca57, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:14.171782 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9195149a-34d1-4aef-8ca0-ece995ea3538 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:14.193680 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1020/2035] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:27:14 2022] PUT /identity/v3/projects/da29f394f9a3424cabfb2344e894e5e5/users/10b17dfb662e42d288036d00461eca57/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:14.204378 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb507db-242b-4262-bec4-1543e374d006 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:14.204666 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb507db-242b-4262-bec4-1543e374d006 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:14.204942 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb507db-242b-4262-bec4-1543e374d006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:14.234194 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-ddb507db-242b-4262-bec4-1543e374d006 None None] MFA Rules not processed for user `10b17dfb662e42d288036d00461eca57`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:14.235458 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-ddb507db-242b-4262-bec4-1543e374d006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:14.306862 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1016/2036] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:27:14.308049 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195149a-34d1-4aef-8ca0-ece995ea3538 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:14.374095 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195149a-34d1-4aef-8ca0-ece995ea3538 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:14.379120 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8104e802-a2b2-4210-93bb-6d78713eac60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:14.379731 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104e802-a2b2-4210-93bb-6d78713eac60 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:14.380013 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104e802-a2b2-4210-93bb-6d78713eac60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:14.380291 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104e802-a2b2-4210-93bb-6d78713eac60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:14.380930 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8104e802-a2b2-4210-93bb-6d78713eac60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:14.383444 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8104e802-a2b2-4210-93bb-6d78713eac60 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:14.417312 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1021/2037] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:27:14 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:27:14.418412 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb507db-242b-4262-bec4-1543e374d006 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:14.430835 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb507db-242b-4262-bec4-1543e374d006 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:14.550224 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce03dd7-db33-44de-8334-6c4731e82536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:14.551621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce03dd7-db33-44de-8334-6c4731e82536 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:14.553044 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce03dd7-db33-44de-8334-6c4731e82536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:14.553141 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce03dd7-db33-44de-8334-6c4731e82536 admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:14.554658 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce03dd7-db33-44de-8334-6c4731e82536 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:14.558052 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce03dd7-db33-44de-8334-6c4731e82536 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:14.618978 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1017/2038] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:27:14 2022] POST /identity/v3/users => generated 295 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:27:14.623628 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8104e802-a2b2-4210-93bb-6d78713eac60 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:14.636953 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8104e802-a2b2-4210-93bb-6d78713eac60 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:14.641840 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff878da-df1a-4004-9eb0-01434fa2f15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:14.642448 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff878da-df1a-4004-9eb0-01434fa2f15b admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:14.643261 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff878da-df1a-4004-9eb0-01434fa2f15b admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:14.643261 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff878da-df1a-4004-9eb0-01434fa2f15b admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:14.643665 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff878da-df1a-4004-9eb0-01434fa2f15b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:14.645954 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff878da-df1a-4004-9eb0-01434fa2f15b admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:14.657335 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1022/2039] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:27:14.662213 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce03dd7-db33-44de-8334-6c4731e82536 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:14.679594 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce03dd7-db33-44de-8334-6c4731e82536 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:14.698780 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41eabf4-e9fa-4be7-b4be-86b2431339b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:14.701426 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41eabf4-e9fa-4be7-b4be-86b2431339b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:14.702016 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41eabf4-e9fa-4be7-b4be-86b2431339b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:14.702715 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41eabf4-e9fa-4be7-b4be-86b2431339b8 admin admin] PATH_INFO: `/v3/projects/bad95c06cd95470f8b783a2bd984209e/users/8d70cbabe0d34b60ab2935b3dde29b6b/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:14.727187 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41eabf4-e9fa-4be7-b4be-86b2431339b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bad95c06cd95470f8b783a2bd984209e, user_id=8d70cbabe0d34b60ab2935b3dde29b6b, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:14.731396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41eabf4-e9fa-4be7-b4be-86b2431339b8 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:14.747279 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1018/2040] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:27:14 2022] PUT /identity/v3/projects/bad95c06cd95470f8b783a2bd984209e/users/8d70cbabe0d34b60ab2935b3dde29b6b/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:14.751295 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff878da-df1a-4004-9eb0-01434fa2f15b admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:14.767304 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff878da-df1a-4004-9eb0-01434fa2f15b admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:14.814088 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:14.839160 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d3ed0f-783b-4ab6-8b7f-5a931252d64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:14.839778 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3ed0f-783b-4ab6-8b7f-5a931252d64c admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:14.840090 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3ed0f-783b-4ab6-8b7f-5a931252d64c admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:14.840363 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3ed0f-783b-4ab6-8b7f-5a931252d64c admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:14.840998 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d3ed0f-783b-4ab6-8b7f-5a931252d64c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:14.842616 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d3ed0f-783b-4ab6-8b7f-5a931252d64c admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:14.852518 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1023/2041] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:14 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:27:14.856632 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41eabf4-e9fa-4be7-b4be-86b2431339b8 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:14.867460 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41eabf4-e9fa-4be7-b4be-86b2431339b8 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:14.871958 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd3cc9-4fb7-4a90-b624-a75c8cb9e31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:14.872598 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd3cc9-4fb7-4a90-b624-a75c8cb9e31c admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:14.872876 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd3cc9-4fb7-4a90-b624-a75c8cb9e31c admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:14.873144 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd3cc9-4fb7-4a90-b624-a75c8cb9e31c admin admin] PATH_INFO: `/v3/projects/bad95c06cd95470f8b783a2bd984209e/users/8d70cbabe0d34b60ab2935b3dde29b6b/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:14.875036 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd3cc9-4fb7-4a90-b624-a75c8cb9e31c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bad95c06cd95470f8b783a2bd984209e, user_id=8d70cbabe0d34b60ab2935b3dde29b6b, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:14.877639 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd3cc9-4fb7-4a90-b624-a75c8cb9e31c admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:14.890120 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1019/2042] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:27:14 2022] PUT /identity/v3/projects/bad95c06cd95470f8b783a2bd984209e/users/8d70cbabe0d34b60ab2935b3dde29b6b/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:14.901274 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191af7ba-31d1-4211-a9c5-48807cb36ba1 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:14.901568 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191af7ba-31d1-4211-a9c5-48807cb36ba1 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:14.901844 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191af7ba-31d1-4211-a9c5-48807cb36ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:14.936154 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-191af7ba-31d1-4211-a9c5-48807cb36ba1 None None] MFA Rules not processed for user `8d70cbabe0d34b60ab2935b3dde29b6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:14.937557 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-191af7ba-31d1-4211-a9c5-48807cb36ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:15.013915 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1024/2043] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:14 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:27:15.025531 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd3cc9-4fb7-4a90-b624-a75c8cb9e31c admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:15.060675 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:15.082807 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd3cc9-4fb7-4a90-b624-a75c8cb9e31c admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:15.086363 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2948faa1-9ebc-4c93-a256-ce2cd34a80cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:15.087008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2948faa1-9ebc-4c93-a256-ce2cd34a80cf service cinder] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:15.087288 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2948faa1-9ebc-4c93-a256-ce2cd34a80cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:15.087558 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2948faa1-9ebc-4c93-a256-ce2cd34a80cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:15.143429 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2948faa1-9ebc-4c93-a256-ce2cd34a80cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:15.144924 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2948faa1-9ebc-4c93-a256-ce2cd34a80cf service cinder] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:15.153975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1020/2044] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:27:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:27:16.336472 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191af7ba-31d1-4211-a9c5-48807cb36ba1 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:16.345847 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191af7ba-31d1-4211-a9c5-48807cb36ba1 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:16.349414 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32479bba-e7ec-4572-8f5d-731aeb964224 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:16.350009 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32479bba-e7ec-4572-8f5d-731aeb964224 service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:16.350285 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32479bba-e7ec-4572-8f5d-731aeb964224 service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:16.350554 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32479bba-e7ec-4572-8f5d-731aeb964224 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:16.352411 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-32479bba-e7ec-4572-8f5d-731aeb964224 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:16.426337 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32479bba-e7ec-4572-8f5d-731aeb964224 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:16.427975 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32479bba-e7ec-4572-8f5d-731aeb964224 service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:16.440521 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1025/2045] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:27:16 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:27:17.760930 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2948faa1-9ebc-4c93-a256-ce2cd34a80cf service cinder] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:17.769705 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2948faa1-9ebc-4c93-a256-ce2cd34a80cf service cinder] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:17.773425 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc15c9d-d2e9-485c-89ff-3a368e146753 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:17.774155 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc15c9d-d2e9-485c-89ff-3a368e146753 service cinder] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:17.774412 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc15c9d-d2e9-485c-89ff-3a368e146753 service cinder] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:17.774769 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc15c9d-d2e9-485c-89ff-3a368e146753 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:17.840524 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc15c9d-d2e9-485c-89ff-3a368e146753 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:17.842116 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc15c9d-d2e9-485c-89ff-3a368e146753 service cinder] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:17.851287 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1021/2046] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:27:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:27:20.685631 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32479bba-e7ec-4572-8f5d-731aeb964224 service cinder] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:20.694712 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32479bba-e7ec-4572-8f5d-731aeb964224 service cinder] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:20.698494 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bb24fe-ed9f-461f-8bec-cd5950867ea7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:20.699143 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb24fe-ed9f-461f-8bec-cd5950867ea7 service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:20.699417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb24fe-ed9f-461f-8bec-cd5950867ea7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:20.699689 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb24fe-ed9f-461f-8bec-cd5950867ea7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:20.708653 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bb24fe-ed9f-461f-8bec-cd5950867ea7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:20.710850 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bb24fe-ed9f-461f-8bec-cd5950867ea7 service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:20.721255 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1026/2047] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:27:20 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:27:27.847097 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=61 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:30.041408 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc15c9d-d2e9-485c-89ff-3a368e146753 service cinder] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:30.126466 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc15c9d-d2e9-485c-89ff-3a368e146753 service cinder] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:30.131067 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cbe5b9-9f12-40f5-8be3-13b80f788410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:30.131693 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbe5b9-9f12-40f5-8be3-13b80f788410 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:30.132487 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbe5b9-9f12-40f5-8be3-13b80f788410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:30.132755 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbe5b9-9f12-40f5-8be3-13b80f788410 admin admin] PATH_INFO: `/v3/users/ad2f391e9d6448b09298892f281af639` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:30.133891 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cbe5b9-9f12-40f5-8be3-13b80f788410 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad2f391e9d6448b09298892f281af639)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:30.135871 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cbe5b9-9f12-40f5-8be3-13b80f788410 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:30.192735 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-61cbe5b9-9f12-40f5-8be3-13b80f788410 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad2f391e9d6448b09298892f281af639'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:30.193644 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-61cbe5b9-9f12-40f5-8be3-13b80f788410 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad2f391e9d6448b09298892f281af639'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:30.201780 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-61cbe5b9-9f12-40f5-8be3-13b80f788410 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad2f391e9d6448b09298892f281af639'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:30.202878 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-61cbe5b9-9f12-40f5-8be3-13b80f788410 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad2f391e9d6448b09298892f281af639'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:30.220635 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-61cbe5b9-9f12-40f5-8be3-13b80f788410 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad2f391e9d6448b09298892f281af639'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:30.236356 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1022/2048] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:27:30 2022] DELETE /identity/v3/users/ad2f391e9d6448b09298892f281af639 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:30.241330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95ecc8-62ae-4426-adc4-298cf576bc30 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:30.241814 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95ecc8-62ae-4426-adc4-298cf576bc30 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:30.241935 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95ecc8-62ae-4426-adc4-298cf576bc30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:30.262752 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-3f95ecc8-62ae-4426-adc4-298cf576bc30 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:30.264048 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-3f95ecc8-62ae-4426-adc4-298cf576bc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:30.373057 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:30.406843 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1027/2049] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:27:30.418641 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cbe5b9-9f12-40f5-8be3-13b80f788410 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:30.421456 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-61cbe5b9-9f12-40f5-8be3-13b80f788410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:30.553478 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cbe5b9-9f12-40f5-8be3-13b80f788410 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:30.559315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfad40-2a41-4ce5-b547-647f8442d262 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:30.560171 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfad40-2a41-4ce5-b547-647f8442d262 service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:30.560777 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfad40-2a41-4ce5-b547-647f8442d262 service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:30.561154 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfad40-2a41-4ce5-b547-647f8442d262 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:30.575707 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bfad40-2a41-4ce5-b547-647f8442d262 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:30.577643 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bfad40-2a41-4ce5-b547-647f8442d262 service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:30.595663 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1023/2050] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:27:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:27:31.056085 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f95ecc8-62ae-4426-adc4-298cf576bc30 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:31.057054 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-3f95ecc8-62ae-4426-adc4-298cf576bc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:31.071036 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f95ecc8-62ae-4426-adc4-298cf576bc30 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:31.075762 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54f481-3a11-47c8-b30e-4f1b36ec28bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:31.076608 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54f481-3a11-47c8-b30e-4f1b36ec28bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:31.077122 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54f481-3a11-47c8-b30e-4f1b36ec28bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:31.077582 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54f481-3a11-47c8-b30e-4f1b36ec28bf admin admin] PATH_INFO: `/v3/projects/46aa71a8a6414987bec0c395b23cc1bd` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:31.078949 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f54f481-3a11-47c8-b30e-4f1b36ec28bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=46aa71a8a6414987bec0c395b23cc1bd)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:31.080972 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f54f481-3a11-47c8-b30e-4f1b36ec28bf admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:31.142617 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-4f54f481-3a11-47c8-b30e-4f1b36ec28bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46aa71a8a6414987bec0c395b23cc1bd', 'request_id': 'req-4f54f481-3a11-47c8-b30e-4f1b36ec28bf'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:31.149847 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-4f54f481-3a11-47c8-b30e-4f1b36ec28bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46aa71a8a6414987bec0c395b23cc1bd', 'request_id': 'req-4f54f481-3a11-47c8-b30e-4f1b36ec28bf'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:31.157094 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-4f54f481-3a11-47c8-b30e-4f1b36ec28bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46aa71a8a6414987bec0c395b23cc1bd', 'request_id': 'req-4f54f481-3a11-47c8-b30e-4f1b36ec28bf'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:31.173552 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-4f54f481-3a11-47c8-b30e-4f1b36ec28bf admin admin] The token cache is being invalidate because project 46aa71a8a6414987bec0c395b23cc1bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:27:31.174117 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-4f54f481-3a11-47c8-b30e-4f1b36ec28bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:31.175026 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-4f54f481-3a11-47c8-b30e-4f1b36ec28bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:31.180554 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1028/2051] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:27:31 2022] DELETE /identity/v3/projects/46aa71a8a6414987bec0c395b23cc1bd => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:31.181947 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfad40-2a41-4ce5-b547-647f8442d262 service neutron] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:31.195334 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfad40-2a41-4ce5-b547-647f8442d262 service neutron] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:31.218495 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:31.269024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf130b-ec6a-4332-8dda-176b84b80811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:31.270143 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf130b-ec6a-4332-8dda-176b84b80811 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:31.270607 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf130b-ec6a-4332-8dda-176b84b80811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:31.271217 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf130b-ec6a-4332-8dda-176b84b80811 admin admin] PATH_INFO: `/v3/users/080a54cdffc24f7fade20dfcb7b2bffe` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:31.273287 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abf130b-ec6a-4332-8dda-176b84b80811 admin admin] RBAC: Authorizing `identity:delete_user(user_id=080a54cdffc24f7fade20dfcb7b2bffe)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:31.275714 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abf130b-ec6a-4332-8dda-176b84b80811 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:31.346789 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-9abf130b-ec6a-4332-8dda-176b84b80811 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '080a54cdffc24f7fade20dfcb7b2bffe'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:31.347680 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-9abf130b-ec6a-4332-8dda-176b84b80811 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '080a54cdffc24f7fade20dfcb7b2bffe'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:31.357245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-9abf130b-ec6a-4332-8dda-176b84b80811 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '080a54cdffc24f7fade20dfcb7b2bffe'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:31.358439 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-9abf130b-ec6a-4332-8dda-176b84b80811 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '080a54cdffc24f7fade20dfcb7b2bffe'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:31.373112 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-9abf130b-ec6a-4332-8dda-176b84b80811 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '080a54cdffc24f7fade20dfcb7b2bffe'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:31.399854 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1024/2052] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:27:31 2022] DELETE /identity/v3/users/080a54cdffc24f7fade20dfcb7b2bffe => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:31.761851 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54f481-3a11-47c8-b30e-4f1b36ec28bf admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:31.763388 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-4f54f481-3a11-47c8-b30e-4f1b36ec28bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:31.791781 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54f481-3a11-47c8-b30e-4f1b36ec28bf admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:31.851645 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a7cd72-b7ae-4a6c-b083-b9470dd83355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:31.852636 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7cd72-b7ae-4a6c-b083-b9470dd83355 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:31.852921 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7cd72-b7ae-4a6c-b083-b9470dd83355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:31.853191 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7cd72-b7ae-4a6c-b083-b9470dd83355 admin admin] PATH_INFO: `/v3/projects/079998319ccc4f73a169e24dc5afadde` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:31.854348 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a7cd72-b7ae-4a6c-b083-b9470dd83355 admin admin] RBAC: Authorizing `identity:delete_project(project_id=079998319ccc4f73a169e24dc5afadde)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:31.856256 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a7cd72-b7ae-4a6c-b083-b9470dd83355 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:31.920209 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d3a7cd72-b7ae-4a6c-b083-b9470dd83355 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '079998319ccc4f73a169e24dc5afadde', 'request_id': 'req-d3a7cd72-b7ae-4a6c-b083-b9470dd83355'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:31.931075 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d3a7cd72-b7ae-4a6c-b083-b9470dd83355 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '079998319ccc4f73a169e24dc5afadde', 'request_id': 'req-d3a7cd72-b7ae-4a6c-b083-b9470dd83355'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:31.943486 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d3a7cd72-b7ae-4a6c-b083-b9470dd83355 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '079998319ccc4f73a169e24dc5afadde', 'request_id': 'req-d3a7cd72-b7ae-4a6c-b083-b9470dd83355'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:31.952792 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:31.977891 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d3a7cd72-b7ae-4a6c-b083-b9470dd83355 admin admin] The token cache is being invalidate because project 079998319ccc4f73a169e24dc5afadde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:27:31.978256 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d3a7cd72-b7ae-4a6c-b083-b9470dd83355 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:31.979009 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d3a7cd72-b7ae-4a6c-b083-b9470dd83355 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:31.983774 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1029/2053] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:27:31 2022] DELETE /identity/v3/projects/079998319ccc4f73a169e24dc5afadde => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:31.984626 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf130b-ec6a-4332-8dda-176b84b80811 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:31.986098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-9abf130b-ec6a-4332-8dda-176b84b80811 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:32.002684 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf130b-ec6a-4332-8dda-176b84b80811 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:32.068377 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cf4801-6a4c-4e11-ad64-b835e4af76cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:32.069326 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf4801-6a4c-4e11-ad64-b835e4af76cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:32.070129 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf4801-6a4c-4e11-ad64-b835e4af76cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:32.070405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf4801-6a4c-4e11-ad64-b835e4af76cd admin admin] PATH_INFO: `/v3/users/052c60814af948ce913db19f1756ef5a` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:32.072051 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cf4801-6a4c-4e11-ad64-b835e4af76cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=052c60814af948ce913db19f1756ef5a)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:32.074027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cf4801-6a4c-4e11-ad64-b835e4af76cd admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:32.119807 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:32.123362 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-32cf4801-6a4c-4e11-ad64-b835e4af76cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '052c60814af948ce913db19f1756ef5a'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:32.124304 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-32cf4801-6a4c-4e11-ad64-b835e4af76cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '052c60814af948ce913db19f1756ef5a'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:32.134291 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-32cf4801-6a4c-4e11-ad64-b835e4af76cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '052c60814af948ce913db19f1756ef5a'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:32.135003 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-32cf4801-6a4c-4e11-ad64-b835e4af76cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '052c60814af948ce913db19f1756ef5a'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:32.149313 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-32cf4801-6a4c-4e11-ad64-b835e4af76cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '052c60814af948ce913db19f1756ef5a'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:32.167407 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1025/2054] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:27:31 2022] DELETE /identity/v3/users/052c60814af948ce913db19f1756ef5a => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:32.468873 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a7cd72-b7ae-4a6c-b083-b9470dd83355 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:32.470337 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-d3a7cd72-b7ae-4a6c-b083-b9470dd83355 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:32.501429 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a7cd72-b7ae-4a6c-b083-b9470dd83355 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:32.581198 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557ca39c-937d-4e00-a394-8432625952c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:32.582316 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ca39c-937d-4e00-a394-8432625952c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:32.582806 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ca39c-937d-4e00-a394-8432625952c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:32.583309 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ca39c-937d-4e00-a394-8432625952c3 admin admin] PATH_INFO: `/v3/projects/4684c7f3a5b947b88586b1250bb37885` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:32.585716 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557ca39c-937d-4e00-a394-8432625952c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4684c7f3a5b947b88586b1250bb37885)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:32.587891 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557ca39c-937d-4e00-a394-8432625952c3 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:32.672439 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-557ca39c-937d-4e00-a394-8432625952c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4684c7f3a5b947b88586b1250bb37885', 'request_id': 'req-557ca39c-937d-4e00-a394-8432625952c3'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:32.684890 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-557ca39c-937d-4e00-a394-8432625952c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4684c7f3a5b947b88586b1250bb37885', 'request_id': 'req-557ca39c-937d-4e00-a394-8432625952c3'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:32.695886 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-557ca39c-937d-4e00-a394-8432625952c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4684c7f3a5b947b88586b1250bb37885', 'request_id': 'req-557ca39c-937d-4e00-a394-8432625952c3'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:32.714013 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-557ca39c-937d-4e00-a394-8432625952c3 admin admin] The token cache is being invalidate because project 4684c7f3a5b947b88586b1250bb37885 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:27:32.714459 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-557ca39c-937d-4e00-a394-8432625952c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:32.715211 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-557ca39c-937d-4e00-a394-8432625952c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:32.717268 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1030/2055] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:27:32 2022] DELETE /identity/v3/projects/4684c7f3a5b947b88586b1250bb37885 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:32.723604 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85def877-d9c1-4da8-9b12-0a534d2bbd01 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:32.723604 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85def877-d9c1-4da8-9b12-0a534d2bbd01 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:32.724067 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85def877-d9c1-4da8-9b12-0a534d2bbd01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:32.752300 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-85def877-d9c1-4da8-9b12-0a534d2bbd01 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:32.754524 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-85def877-d9c1-4da8-9b12-0a534d2bbd01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:32.826649 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557ca39c-937d-4e00-a394-8432625952c3 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:32.828817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-557ca39c-937d-4e00-a394-8432625952c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:32.856672 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1026/2056] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:27:32.865640 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279a996-f8d2-4031-8780-10a75bab8590 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:32.866758 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279a996-f8d2-4031-8780-10a75bab8590 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:32.867204 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279a996-f8d2-4031-8780-10a75bab8590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:32.892788 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-f279a996-f8d2-4031-8780-10a75bab8590 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:32.894273 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-f279a996-f8d2-4031-8780-10a75bab8590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:32.906251 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1027/2057] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:27:32.925480 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f279a996-f8d2-4031-8780-10a75bab8590 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:32.937332 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f279a996-f8d2-4031-8780-10a75bab8590 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:32.942359 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f5fe9e-f362-4da0-b430-c9aa9744423b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:32.943081 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f5fe9e-f362-4da0-b430-c9aa9744423b admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:32.943432 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f5fe9e-f362-4da0-b430-c9aa9744423b admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:32.943778 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f5fe9e-f362-4da0-b430-c9aa9744423b admin admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:32.944689 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f5fe9e-f362-4da0-b430-c9aa9744423b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:27:32.945057 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f5fe9e-f362-4da0-b430-c9aa9744423b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:32.946944 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f5fe9e-f362-4da0-b430-c9aa9744423b admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:32.961722 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1028/2058] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:27:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:27:32.966392 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f5fe9e-f362-4da0-b430-c9aa9744423b admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:32.969111 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557ca39c-937d-4e00-a394-8432625952c3 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:32.979630 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f5fe9e-f362-4da0-b430-c9aa9744423b admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:32.980264 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83575e84-8dbd-48bb-9266-dfa25bce82b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:32.981347 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83575e84-8dbd-48bb-9266-dfa25bce82b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:32.981693 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83575e84-8dbd-48bb-9266-dfa25bce82b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:32.982032 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83575e84-8dbd-48bb-9266-dfa25bce82b1 admin admin] PATH_INFO: `/v3/users/8af199617ed5413b96728d1171892bbf` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:32.983551 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83575e84-8dbd-48bb-9266-dfa25bce82b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8af199617ed5413b96728d1171892bbf)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:32.985326 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373d0dbd-e407-48ca-b443-a522675f162f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:32.986155 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d0dbd-e407-48ca-b443-a522675f162f admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:32.986499 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83575e84-8dbd-48bb-9266-dfa25bce82b1 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:32.986611 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d0dbd-e407-48ca-b443-a522675f162f admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:32.986975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d0dbd-e407-48ca-b443-a522675f162f admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:32.987918 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373d0dbd-e407-48ca-b443-a522675f162f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:32.990702 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373d0dbd-e407-48ca-b443-a522675f162f admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:33.037388 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1029/2059] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:27:32 2022] POST /identity/v3/projects => generated 381 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:27:33.042542 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373d0dbd-e407-48ca-b443-a522675f162f admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:33.043986 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:33.085462 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-83575e84-8dbd-48bb-9266-dfa25bce82b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8af199617ed5413b96728d1171892bbf'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:33.086283 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-83575e84-8dbd-48bb-9266-dfa25bce82b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8af199617ed5413b96728d1171892bbf'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:33.087032 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-83575e84-8dbd-48bb-9266-dfa25bce82b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8af199617ed5413b96728d1171892bbf'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:33.106275 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-83575e84-8dbd-48bb-9266-dfa25bce82b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8af199617ed5413b96728d1171892bbf'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:33.115416 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-83575e84-8dbd-48bb-9266-dfa25bce82b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8af199617ed5413b96728d1171892bbf'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:33.138705 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1031/2060] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:27:32 2022] DELETE /identity/v3/users/8af199617ed5413b96728d1171892bbf => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:33.140177 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef562ec-1ac8-498e-b1cf-6798cb6ff3fb None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:33.140667 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef562ec-1ac8-498e-b1cf-6798cb6ff3fb None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:33.140950 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef562ec-1ac8-498e-b1cf-6798cb6ff3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:33.146442 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373d0dbd-e407-48ca-b443-a522675f162f admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:33.166840 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-eef562ec-1ac8-498e-b1cf-6798cb6ff3fb None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:33.168160 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-eef562ec-1ac8-498e-b1cf-6798cb6ff3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:33.175110 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:33.217578 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6fb5ce-e186-411d-9240-5a3772270f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:33.218194 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6fb5ce-e186-411d-9240-5a3772270f4f admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:33.218472 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6fb5ce-e186-411d-9240-5a3772270f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:33.218741 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6fb5ce-e186-411d-9240-5a3772270f4f admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:33.219412 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6fb5ce-e186-411d-9240-5a3772270f4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:33.221419 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6fb5ce-e186-411d-9240-5a3772270f4f admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:33.239798 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1032/2061] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:27:33.250164 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef562ec-1ac8-498e-b1cf-6798cb6ff3fb None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:33.251642 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-eef562ec-1ac8-498e-b1cf-6798cb6ff3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:33.258724 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1030/2062] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:27:33 2022] POST /identity/v3/users => generated 291 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:27:33.260007 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6fb5ce-e186-411d-9240-5a3772270f4f admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:33.261626 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-3c6fb5ce-e186-411d-9240-5a3772270f4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:33.273826 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6fb5ce-e186-411d-9240-5a3772270f4f admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:33.278423 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a5710d-841d-405a-8798-df16ea1ba7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:33.279039 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5710d-841d-405a-8798-df16ea1ba7ca admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:33.279322 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5710d-841d-405a-8798-df16ea1ba7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:33.279589 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5710d-841d-405a-8798-df16ea1ba7ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:33.280332 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a5710d-841d-405a-8798-df16ea1ba7ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:33.281911 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a5710d-841d-405a-8798-df16ea1ba7ca admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:33.292498 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1031/2063] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 14:27:33.296185 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a5710d-841d-405a-8798-df16ea1ba7ca admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:33.306155 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a5710d-841d-405a-8798-df16ea1ba7ca admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:33.310748 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba041b5-4804-449d-9dc0-21e9528233dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:33.311358 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba041b5-4804-449d-9dc0-21e9528233dc admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:33.311663 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba041b5-4804-449d-9dc0-21e9528233dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:33.311929 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba041b5-4804-449d-9dc0-21e9528233dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:33.312462 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef562ec-1ac8-498e-b1cf-6798cb6ff3fb None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:33.312580 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba041b5-4804-449d-9dc0-21e9528233dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:33.314198 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba041b5-4804-449d-9dc0-21e9528233dc admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:33.318834 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf55114-4da6-4792-87cc-b84c555d7406 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:33.320257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf55114-4da6-4792-87cc-b84c555d7406 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:33.320576 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf55114-4da6-4792-87cc-b84c555d7406 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:33.320873 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf55114-4da6-4792-87cc-b84c555d7406 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:33.328538 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1032/2064] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:27:33.329420 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba041b5-4804-449d-9dc0-21e9528233dc admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:33.331226 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf55114-4da6-4792-87cc-b84c555d7406 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:33.332671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf55114-4da6-4792-87cc-b84c555d7406 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:33.341470 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba041b5-4804-449d-9dc0-21e9528233dc admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:33.344543 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1033/2065] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:27:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 14:27:33.348345 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6ca1c6-a7da-433d-b5f6-c3104fb9bd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:33.349116 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ca1c6-a7da-433d-b5f6-c3104fb9bd20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:33.349604 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ca1c6-a7da-433d-b5f6-c3104fb9bd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:33.349825 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ca1c6-a7da-433d-b5f6-c3104fb9bd20 admin admin] PATH_INFO: `/v3/projects/27e584f45faa47afa4ced9db997f4e4a/users/d381a365e0744834b716fa6b0ea18d3d/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:33.368458 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6ca1c6-a7da-433d-b5f6-c3104fb9bd20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27e584f45faa47afa4ced9db997f4e4a, user_id=d381a365e0744834b716fa6b0ea18d3d, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:33.371152 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6ca1c6-a7da-433d-b5f6-c3104fb9bd20 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:33.387160 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1033/2066] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:27:33 2022] PUT /identity/v3/projects/27e584f45faa47afa4ced9db997f4e4a/users/d381a365e0744834b716fa6b0ea18d3d/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:27:33.392706 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c4db5-6e56-47bf-ac9e-1b5fa536fd84 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:33.397568 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c4db5-6e56-47bf-ac9e-1b5fa536fd84 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:33.397568 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c4db5-6e56-47bf-ac9e-1b5fa536fd84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:33.399152 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6ca1c6-a7da-433d-b5f6-c3104fb9bd20 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:33.401907 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-aa6ca1c6-a7da-433d-b5f6-c3104fb9bd20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:33.416684 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-776c4db5-6e56-47bf-ac9e-1b5fa536fd84 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:33.418113 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-776c4db5-6e56-47bf-ac9e-1b5fa536fd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:33.474487 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:33.498866 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1034/2067] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:27:33.505971 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6ca1c6-a7da-433d-b5f6-c3104fb9bd20 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:33.510280 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c4db5-6e56-47bf-ac9e-1b5fa536fd84 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:33.511646 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed062591-4eb8-4fe2-828a-5bc09bcba0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:33.512036 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-776c4db5-6e56-47bf-ac9e-1b5fa536fd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:33.512424 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed062591-4eb8-4fe2-828a-5bc09bcba0a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:33.512766 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed062591-4eb8-4fe2-828a-5bc09bcba0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:33.513104 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed062591-4eb8-4fe2-828a-5bc09bcba0a4 admin admin] PATH_INFO: `/v3/users/3a2917017d544048921c4c69ed018af0` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:33.514357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed062591-4eb8-4fe2-828a-5bc09bcba0a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a2917017d544048921c4c69ed018af0)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:33.516566 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed062591-4eb8-4fe2-828a-5bc09bcba0a4 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:33.576533 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-ed062591-4eb8-4fe2-828a-5bc09bcba0a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a2917017d544048921c4c69ed018af0'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:33.577285 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-ed062591-4eb8-4fe2-828a-5bc09bcba0a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a2917017d544048921c4c69ed018af0'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:33.591583 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-ed062591-4eb8-4fe2-828a-5bc09bcba0a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a2917017d544048921c4c69ed018af0'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:33.593147 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-ed062591-4eb8-4fe2-828a-5bc09bcba0a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a2917017d544048921c4c69ed018af0'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:33.601896 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c4db5-6e56-47bf-ac9e-1b5fa536fd84 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:33.606705 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e2b38e-1933-4464-89bb-1a80116a6fda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:33.607432 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2b38e-1933-4464-89bb-1a80116a6fda service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:33.607777 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2b38e-1933-4464-89bb-1a80116a6fda service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:33.608112 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2b38e-1933-4464-89bb-1a80116a6fda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:33.609309 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:33.610868 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-ed062591-4eb8-4fe2-828a-5bc09bcba0a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a2917017d544048921c4c69ed018af0'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:33.611577 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-b8e2b38e-1933-4464-89bb-1a80116a6fda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:33.629932 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1034/2068] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:27:33 2022] DELETE /identity/v3/users/3a2917017d544048921c4c69ed018af0 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:33.633683 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e2b38e-1933-4464-89bb-1a80116a6fda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:33.635078 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d8fb6-20ff-4269-9c98-591be3b44968 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:33.635366 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d8fb6-20ff-4269-9c98-591be3b44968 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:33.635643 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d8fb6-20ff-4269-9c98-591be3b44968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:33.636102 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e2b38e-1933-4464-89bb-1a80116a6fda service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:33.654442 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1035/2069] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:27:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:27:33.668345 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-1d2d8fb6-20ff-4269-9c98-591be3b44968 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:33.670104 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-1d2d8fb6-20ff-4269-9c98-591be3b44968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:33.677268 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e2b38e-1933-4464-89bb-1a80116a6fda service neutron] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:33.680174 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-b8e2b38e-1933-4464-89bb-1a80116a6fda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:33.702774 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e2b38e-1933-4464-89bb-1a80116a6fda service neutron] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:33.786766 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1035/2070] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:27:33.798390 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d8fb6-20ff-4269-9c98-591be3b44968 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:33.800388 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-1d2d8fb6-20ff-4269-9c98-591be3b44968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:33.813566 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39492d67-c632-45a9-b83c-2b7503561659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:33.820986 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39492d67-c632-45a9-b83c-2b7503561659 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:33.820986 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39492d67-c632-45a9-b83c-2b7503561659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:33.820986 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39492d67-c632-45a9-b83c-2b7503561659 admin admin] PATH_INFO: `/v3/projects/eda57b62d6bc44928579c7a35f3e251d` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:33.823068 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39492d67-c632-45a9-b83c-2b7503561659 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eda57b62d6bc44928579c7a35f3e251d)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:33.825776 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39492d67-c632-45a9-b83c-2b7503561659 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:33.895324 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d8fb6-20ff-4269-9c98-591be3b44968 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:33.902315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180b1aaf-b3e6-4031-9c7d-23ed52a49020 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:33.902477 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b1aaf-b3e6-4031-9c7d-23ed52a49020 service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:33.902770 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b1aaf-b3e6-4031-9c7d-23ed52a49020 service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:33.903170 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b1aaf-b3e6-4031-9c7d-23ed52a49020 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:33.920957 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180b1aaf-b3e6-4031-9c7d-23ed52a49020 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:33.922794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180b1aaf-b3e6-4031-9c7d-23ed52a49020 service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:33.937817 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1036/2071] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:27:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:27:33.964651 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-39492d67-c632-45a9-b83c-2b7503561659 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eda57b62d6bc44928579c7a35f3e251d', 'request_id': 'req-39492d67-c632-45a9-b83c-2b7503561659'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:33.974403 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-39492d67-c632-45a9-b83c-2b7503561659 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eda57b62d6bc44928579c7a35f3e251d', 'request_id': 'req-39492d67-c632-45a9-b83c-2b7503561659'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:33.990743 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-39492d67-c632-45a9-b83c-2b7503561659 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eda57b62d6bc44928579c7a35f3e251d', 'request_id': 'req-39492d67-c632-45a9-b83c-2b7503561659'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:34.021716 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-39492d67-c632-45a9-b83c-2b7503561659 admin admin] The token cache is being invalidate because project eda57b62d6bc44928579c7a35f3e251d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:27:34.022937 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-39492d67-c632-45a9-b83c-2b7503561659 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:34.024059 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-39492d67-c632-45a9-b83c-2b7503561659 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:34.026991 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1036/2072] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:27:33 2022] DELETE /identity/v3/projects/eda57b62d6bc44928579c7a35f3e251d => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:34.031187 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180b1aaf-b3e6-4031-9c7d-23ed52a49020 service neutron] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:34.045511 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180b1aaf-b3e6-4031-9c7d-23ed52a49020 service neutron] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:34.172628 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855efdd8-04f5-44a0-af4a-928e08760c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:34.173262 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855efdd8-04f5-44a0-af4a-928e08760c96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:34.173551 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855efdd8-04f5-44a0-af4a-928e08760c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:34.173867 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855efdd8-04f5-44a0-af4a-928e08760c96 admin admin] PATH_INFO: `/v3/users/2e70ba3e1f6a42e3bc3dec61232c6c02` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:34.176182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855efdd8-04f5-44a0-af4a-928e08760c96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e70ba3e1f6a42e3bc3dec61232c6c02)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:34.191029 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855efdd8-04f5-44a0-af4a-928e08760c96 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:34.204631 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39492d67-c632-45a9-b83c-2b7503561659 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:34.206560 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-39492d67-c632-45a9-b83c-2b7503561659 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:34.236712 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39492d67-c632-45a9-b83c-2b7503561659 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:34.246007 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd6cce-7d6c-4b8e-8bac-53bd74d374b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:34.246748 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd6cce-7d6c-4b8e-8bac-53bd74d374b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:34.249392 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd6cce-7d6c-4b8e-8bac-53bd74d374b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:34.250126 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd6cce-7d6c-4b8e-8bac-53bd74d374b6 admin admin] PATH_INFO: `/v3/users/e77d048ffd33407aae0dbbd04e2c30ea` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:34.253462 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dd6cce-7d6c-4b8e-8bac-53bd74d374b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e77d048ffd33407aae0dbbd04e2c30ea)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:34.261845 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dd6cce-7d6c-4b8e-8bac-53bd74d374b6 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:34.297191 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:34.312647 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:34.341113 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-855efdd8-04f5-44a0-af4a-928e08760c96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e70ba3e1f6a42e3bc3dec61232c6c02'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:34.341832 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-855efdd8-04f5-44a0-af4a-928e08760c96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e70ba3e1f6a42e3bc3dec61232c6c02'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:34.355344 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-855efdd8-04f5-44a0-af4a-928e08760c96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e70ba3e1f6a42e3bc3dec61232c6c02'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:34.356456 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-855efdd8-04f5-44a0-af4a-928e08760c96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e70ba3e1f6a42e3bc3dec61232c6c02'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:34.387411 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-855efdd8-04f5-44a0-af4a-928e08760c96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e70ba3e1f6a42e3bc3dec61232c6c02'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:34.390504 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-b6dd6cce-7d6c-4b8e-8bac-53bd74d374b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e77d048ffd33407aae0dbbd04e2c30ea'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:34.391755 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-b6dd6cce-7d6c-4b8e-8bac-53bd74d374b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e77d048ffd33407aae0dbbd04e2c30ea'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:34.392383 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-b6dd6cce-7d6c-4b8e-8bac-53bd74d374b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e77d048ffd33407aae0dbbd04e2c30ea'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:34.405754 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1037/2073] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:27:34 2022] DELETE /identity/v3/users/2e70ba3e1f6a42e3bc3dec61232c6c02 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:27:34.406743 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855efdd8-04f5-44a0-af4a-928e08760c96 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:34.408620 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-b6dd6cce-7d6c-4b8e-8bac-53bd74d374b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e77d048ffd33407aae0dbbd04e2c30ea'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:34.408866 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-855efdd8-04f5-44a0-af4a-928e08760c96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:34.424621 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-b6dd6cce-7d6c-4b8e-8bac-53bd74d374b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e77d048ffd33407aae0dbbd04e2c30ea'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:34.456637 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1037/2074] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:27:34 2022] DELETE /identity/v3/users/e77d048ffd33407aae0dbbd04e2c30ea => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:34.458147 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8671c7c-6a4a-4de3-83ee-d89ac689653c None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:34.458431 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8671c7c-6a4a-4de3-83ee-d89ac689653c None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:34.458699 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8671c7c-6a4a-4de3-83ee-d89ac689653c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:34.485612 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-d8671c7c-6a4a-4de3-83ee-d89ac689653c None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:34.486857 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-d8671c7c-6a4a-4de3-83ee-d89ac689653c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:34.532091 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855efdd8-04f5-44a0-af4a-928e08760c96 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:34.607083 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1038/2075] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:27:34.627083 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8671c7c-6a4a-4de3-83ee-d89ac689653c None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:34.629493 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-d8671c7c-6a4a-4de3-83ee-d89ac689653c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:34.642363 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533868f3-6e31-490c-bb6b-baec06839f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:34.644611 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533868f3-6e31-490c-bb6b-baec06839f9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:34.645156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533868f3-6e31-490c-bb6b-baec06839f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:34.645715 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533868f3-6e31-490c-bb6b-baec06839f9a admin admin] PATH_INFO: `/v3/users/9c0f213d9cbb4418a759559972ab748d` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:34.650678 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533868f3-6e31-490c-bb6b-baec06839f9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c0f213d9cbb4418a759559972ab748d)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:34.658863 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533868f3-6e31-490c-bb6b-baec06839f9a admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:34.736348 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8671c7c-6a4a-4de3-83ee-d89ac689653c None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:34.744217 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63921f-4a6a-4fcd-87fc-abd0e3904125 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:34.745300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63921f-4a6a-4fcd-87fc-abd0e3904125 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:34.745747 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63921f-4a6a-4fcd-87fc-abd0e3904125 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:34.746145 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63921f-4a6a-4fcd-87fc-abd0e3904125 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:34.752083 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-533868f3-6e31-490c-bb6b-baec06839f9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c0f213d9cbb4418a759559972ab748d'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:34.753002 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-533868f3-6e31-490c-bb6b-baec06839f9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c0f213d9cbb4418a759559972ab748d'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:34.765988 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d63921f-4a6a-4fcd-87fc-abd0e3904125 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:34.767074 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-533868f3-6e31-490c-bb6b-baec06839f9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c0f213d9cbb4418a759559972ab748d'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:34.768227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-533868f3-6e31-490c-bb6b-baec06839f9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c0f213d9cbb4418a759559972ab748d'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:34.768894 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d63921f-4a6a-4fcd-87fc-abd0e3904125 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:34.776926 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:34.790645 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-533868f3-6e31-490c-bb6b-baec06839f9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c0f213d9cbb4418a759559972ab748d'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:34.791260 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1039/2076] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:27:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 14:27:34.814445 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1038/2077] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:27:34 2022] DELETE /identity/v3/users/9c0f213d9cbb4418a759559972ab748d => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:34.821136 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8364e-20f9-452f-a9b9-8839e3068111 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:34.821426 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8364e-20f9-452f-a9b9-8839e3068111 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:34.821763 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8364e-20f9-452f-a9b9-8839e3068111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:34.854329 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-71e8364e-20f9-452f-a9b9-8839e3068111 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:34.855920 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-71e8364e-20f9-452f-a9b9-8839e3068111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:34.872098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533868f3-6e31-490c-bb6b-baec06839f9a admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:34.874510 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-533868f3-6e31-490c-bb6b-baec06839f9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:34.973841 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:35.027829 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1040/2078] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:27:35.028823 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e8364e-20f9-452f-a9b9-8839e3068111 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:35.030521 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-71e8364e-20f9-452f-a9b9-8839e3068111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:35.047620 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e8364e-20f9-452f-a9b9-8839e3068111 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:35.052990 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f155f72e-ae06-41b7-82a0-12ba1c673842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:35.054527 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155f72e-ae06-41b7-82a0-12ba1c673842 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:35.054927 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155f72e-ae06-41b7-82a0-12ba1c673842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:35.055275 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155f72e-ae06-41b7-82a0-12ba1c673842 admin admin] PATH_INFO: `/v3/projects/960b936a0f964349b1641942bbd13848` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:35.056691 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f155f72e-ae06-41b7-82a0-12ba1c673842 admin admin] RBAC: Authorizing `identity:delete_project(project_id=960b936a0f964349b1641942bbd13848)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:35.058773 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f155f72e-ae06-41b7-82a0-12ba1c673842 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:35.067635 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533868f3-6e31-490c-bb6b-baec06839f9a admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:35.073222 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf62e9d9-767d-40f3-8c32-3c89337b9545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:35.073872 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62e9d9-767d-40f3-8c32-3c89337b9545 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:35.074152 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62e9d9-767d-40f3-8c32-3c89337b9545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:35.074427 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62e9d9-767d-40f3-8c32-3c89337b9545 admin admin] PATH_INFO: `/v3/projects/1ef76f16c463457e968dce2749a14fe8` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:35.075593 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf62e9d9-767d-40f3-8c32-3c89337b9545 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ef76f16c463457e968dce2749a14fe8)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:35.078189 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf62e9d9-767d-40f3-8c32-3c89337b9545 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:35.153083 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-bf62e9d9-767d-40f3-8c32-3c89337b9545 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ef76f16c463457e968dce2749a14fe8', 'request_id': 'req-bf62e9d9-767d-40f3-8c32-3c89337b9545'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:35.162767 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-f155f72e-ae06-41b7-82a0-12ba1c673842 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '960b936a0f964349b1641942bbd13848', 'request_id': 'req-f155f72e-ae06-41b7-82a0-12ba1c673842'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:35.164951 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-bf62e9d9-767d-40f3-8c32-3c89337b9545 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ef76f16c463457e968dce2749a14fe8', 'request_id': 'req-bf62e9d9-767d-40f3-8c32-3c89337b9545'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:35.172363 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-f155f72e-ae06-41b7-82a0-12ba1c673842 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '960b936a0f964349b1641942bbd13848', 'request_id': 'req-f155f72e-ae06-41b7-82a0-12ba1c673842'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:35.173990 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-bf62e9d9-767d-40f3-8c32-3c89337b9545 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ef76f16c463457e968dce2749a14fe8', 'request_id': 'req-bf62e9d9-767d-40f3-8c32-3c89337b9545'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:35.184642 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-f155f72e-ae06-41b7-82a0-12ba1c673842 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '960b936a0f964349b1641942bbd13848', 'request_id': 'req-f155f72e-ae06-41b7-82a0-12ba1c673842'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:35.204229 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-bf62e9d9-767d-40f3-8c32-3c89337b9545 admin admin] The token cache is being invalidate because project 1ef76f16c463457e968dce2749a14fe8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:27:35.204541 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-bf62e9d9-767d-40f3-8c32-3c89337b9545 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:35.205313 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-bf62e9d9-767d-40f3-8c32-3c89337b9545 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:35.207264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1039/2079] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:27:34 2022] DELETE /identity/v3/projects/1ef76f16c463457e968dce2749a14fe8 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:35.208058 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf62e9d9-767d-40f3-8c32-3c89337b9545 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:35.211608 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-bf62e9d9-767d-40f3-8c32-3c89337b9545 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:35.215214 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-f155f72e-ae06-41b7-82a0-12ba1c673842 admin admin] The token cache is being invalidate because project 960b936a0f964349b1641942bbd13848 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:27:35.215739 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-f155f72e-ae06-41b7-82a0-12ba1c673842 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:35.216778 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-f155f72e-ae06-41b7-82a0-12ba1c673842 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:35.223289 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1041/2080] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:27:35 2022] DELETE /identity/v3/projects/960b936a0f964349b1641942bbd13848 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:35.224794 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8218d77f-8e73-4227-9291-0db048682343 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:35.225264 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8218d77f-8e73-4227-9291-0db048682343 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:35.226406 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8218d77f-8e73-4227-9291-0db048682343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:35.273567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-8218d77f-8e73-4227-9291-0db048682343 None None] MFA Rules not processed for user `d381a365e0744834b716fa6b0ea18d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:35.275771 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-8218d77f-8e73-4227-9291-0db048682343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:35.294036 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf62e9d9-767d-40f3-8c32-3c89337b9545 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:35.299027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e6151d-cfbd-43bd-9924-052d3b059e5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:35.300067 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6151d-cfbd-43bd-9924-052d3b059e5c service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:35.300536 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6151d-cfbd-43bd-9924-052d3b059e5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:35.301021 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6151d-cfbd-43bd-9924-052d3b059e5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:35.304563 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-70e6151d-cfbd-43bd-9924-052d3b059e5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:35.333118 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:35.387994 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1042/2081] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:27:35.389181 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac7a2f-c156-42f3-a027-ee3f5c4793c8 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:35.389503 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac7a2f-c156-42f3-a027-ee3f5c4793c8 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:35.389771 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac7a2f-c156-42f3-a027-ee3f5c4793c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:35.409345 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e6151d-cfbd-43bd-9924-052d3b059e5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:35.410978 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e6151d-cfbd-43bd-9924-052d3b059e5c service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:35.415458 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-f4ac7a2f-c156-42f3-a027-ee3f5c4793c8 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:35.416680 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-f4ac7a2f-c156-42f3-a027-ee3f5c4793c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:35.441246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1040/2082] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:27:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:27:35.441441 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e6151d-cfbd-43bd-9924-052d3b059e5c service neutron] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:35.445489 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1043/2083] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:27:35.448592 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-70e6151d-cfbd-43bd-9924-052d3b059e5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:35.453638 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac7a2f-c156-42f3-a027-ee3f5c4793c8 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:35.457003 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-f4ac7a2f-c156-42f3-a027-ee3f5c4793c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:35.479756 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e6151d-cfbd-43bd-9924-052d3b059e5c service neutron] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:35.490463 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211ea73b-af10-4f1e-a080-1e57e1ecd890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:35.491923 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ea73b-af10-4f1e-a080-1e57e1ecd890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:35.494006 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ea73b-af10-4f1e-a080-1e57e1ecd890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:35.494371 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ea73b-af10-4f1e-a080-1e57e1ecd890 admin admin] PATH_INFO: `/v3/users/10b17dfb662e42d288036d00461eca57` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:35.497289 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211ea73b-af10-4f1e-a080-1e57e1ecd890 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10b17dfb662e42d288036d00461eca57)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:35.505335 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211ea73b-af10-4f1e-a080-1e57e1ecd890 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:35.566273 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:35.596726 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac7a2f-c156-42f3-a027-ee3f5c4793c8 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:35.605617 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4437e294-d742-4378-9401-dac00ba628ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:35.606251 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437e294-d742-4378-9401-dac00ba628ae service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:35.606537 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437e294-d742-4378-9401-dac00ba628ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:35.607076 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437e294-d742-4378-9401-dac00ba628ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:35.613436 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-211ea73b-af10-4f1e-a080-1e57e1ecd890 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10b17dfb662e42d288036d00461eca57'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:35.615734 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-211ea73b-af10-4f1e-a080-1e57e1ecd890 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10b17dfb662e42d288036d00461eca57'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:35.623735 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4437e294-d742-4378-9401-dac00ba628ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:35.625790 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4437e294-d742-4378-9401-dac00ba628ae service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:35.628642 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-211ea73b-af10-4f1e-a080-1e57e1ecd890 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10b17dfb662e42d288036d00461eca57'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:35.629406 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-211ea73b-af10-4f1e-a080-1e57e1ecd890 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10b17dfb662e42d288036d00461eca57'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:35.638421 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1044/2084] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:27:35 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:27:35.640195 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c01d57-3765-4ebe-bf70-129491613dee None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:35.640725 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c01d57-3765-4ebe-bf70-129491613dee None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:35.641225 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c01d57-3765-4ebe-bf70-129491613dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:35.649880 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-211ea73b-af10-4f1e-a080-1e57e1ecd890 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10b17dfb662e42d288036d00461eca57'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:35.673128 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1041/2085] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:27:35 2022] DELETE /identity/v3/users/10b17dfb662e42d288036d00461eca57 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:35.674050 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211ea73b-af10-4f1e-a080-1e57e1ecd890 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:35.675519 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-211ea73b-af10-4f1e-a080-1e57e1ecd890 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:35.684089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-70c01d57-3765-4ebe-bf70-129491613dee None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:35.685344 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-70c01d57-3765-4ebe-bf70-129491613dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:35.779083 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1045/2086] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:27:35.780624 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c01d57-3765-4ebe-bf70-129491613dee None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:35.801352 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c01d57-3765-4ebe-bf70-129491613dee None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:35.808124 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26fb891-a6cf-4724-9a6d-7ad7235f105b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:35.808942 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fb891-a6cf-4724-9a6d-7ad7235f105b admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:35.809357 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fb891-a6cf-4724-9a6d-7ad7235f105b admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:35.809630 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fb891-a6cf-4724-9a6d-7ad7235f105b admin admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:35.810350 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26fb891-a6cf-4724-9a6d-7ad7235f105b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:27:35.810659 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26fb891-a6cf-4724-9a6d-7ad7235f105b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:35.812636 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26fb891-a6cf-4724-9a6d-7ad7235f105b admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:35.813967 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211ea73b-af10-4f1e-a080-1e57e1ecd890 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:35.834021 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b3eb4b-38e9-4895-b40e-1f1eb8b4d9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:35.835384 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b3eb4b-38e9-4895-b40e-1f1eb8b4d9e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:35.837166 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b3eb4b-38e9-4895-b40e-1f1eb8b4d9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:35.837515 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b3eb4b-38e9-4895-b40e-1f1eb8b4d9e2 admin admin] PATH_INFO: `/v3/projects/fcc17c3a205b441db38b2e70152cdd35` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:35.841050 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b3eb4b-38e9-4895-b40e-1f1eb8b4d9e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcc17c3a205b441db38b2e70152cdd35)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:35.844357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b3eb4b-38e9-4895-b40e-1f1eb8b4d9e2 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:35.845334 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1046/2087] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:27:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:27:35.859025 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26fb891-a6cf-4724-9a6d-7ad7235f105b admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:35.888660 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26fb891-a6cf-4724-9a6d-7ad7235f105b admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:35.893892 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f55aaf5-b796-4268-935a-071d9bc14a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:35.894896 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55aaf5-b796-4268-935a-071d9bc14a9f admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:35.895295 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55aaf5-b796-4268-935a-071d9bc14a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:35.895570 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55aaf5-b796-4268-935a-071d9bc14a9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:35.896505 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f55aaf5-b796-4268-935a-071d9bc14a9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:35.899726 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f55aaf5-b796-4268-935a-071d9bc14a9f admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:35.918425 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:35.969795 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-45b3eb4b-38e9-4895-b40e-1f1eb8b4d9e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcc17c3a205b441db38b2e70152cdd35', 'request_id': 'req-45b3eb4b-38e9-4895-b40e-1f1eb8b4d9e2'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:35.970030 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1047/2088] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:27:35 2022] POST /identity/v3/projects => generated 407 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:27:35.971050 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f55aaf5-b796-4268-935a-071d9bc14a9f admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:35.975709 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-0f55aaf5-b796-4268-935a-071d9bc14a9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:35.980119 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-45b3eb4b-38e9-4895-b40e-1f1eb8b4d9e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcc17c3a205b441db38b2e70152cdd35', 'request_id': 'req-45b3eb4b-38e9-4895-b40e-1f1eb8b4d9e2'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:35.995486 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-45b3eb4b-38e9-4895-b40e-1f1eb8b4d9e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcc17c3a205b441db38b2e70152cdd35', 'request_id': 'req-45b3eb4b-38e9-4895-b40e-1f1eb8b4d9e2'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:36.032631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-45b3eb4b-38e9-4895-b40e-1f1eb8b4d9e2 admin admin] The token cache is being invalidate because project fcc17c3a205b441db38b2e70152cdd35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:27:36.033149 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-45b3eb4b-38e9-4895-b40e-1f1eb8b4d9e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:36.034082 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-45b3eb4b-38e9-4895-b40e-1f1eb8b4d9e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:36.036460 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1042/2089] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:27:35 2022] DELETE /identity/v3/projects/fcc17c3a205b441db38b2e70152cdd35 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:36.040462 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b3eb4b-38e9-4895-b40e-1f1eb8b4d9e2 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:36.044931 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-45b3eb4b-38e9-4895-b40e-1f1eb8b4d9e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:36.124044 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f55aaf5-b796-4268-935a-071d9bc14a9f admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:36.144840 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1bf69b-8dec-4e34-8467-1eccb784bc64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:36.145506 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1bf69b-8dec-4e34-8467-1eccb784bc64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:36.145794 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1bf69b-8dec-4e34-8467-1eccb784bc64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:36.146505 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1bf69b-8dec-4e34-8467-1eccb784bc64 admin admin] PATH_INFO: `/v3/projects/18d5cf0251234c518d493e8a529999d0` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:36.148822 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1bf69b-8dec-4e34-8467-1eccb784bc64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18d5cf0251234c518d493e8a529999d0)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:36.156071 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1bf69b-8dec-4e34-8467-1eccb784bc64 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:36.233520 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-6c1bf69b-8dec-4e34-8467-1eccb784bc64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18d5cf0251234c518d493e8a529999d0', 'request_id': 'req-6c1bf69b-8dec-4e34-8467-1eccb784bc64'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:36.237799 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:36.244314 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-6c1bf69b-8dec-4e34-8467-1eccb784bc64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18d5cf0251234c518d493e8a529999d0', 'request_id': 'req-6c1bf69b-8dec-4e34-8467-1eccb784bc64'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:36.256089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-6c1bf69b-8dec-4e34-8467-1eccb784bc64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18d5cf0251234c518d493e8a529999d0', 'request_id': 'req-6c1bf69b-8dec-4e34-8467-1eccb784bc64'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:36.256306 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b3eb4b-38e9-4895-b40e-1f1eb8b4d9e2 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:36.303017 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-6c1bf69b-8dec-4e34-8467-1eccb784bc64 admin admin] The token cache is being invalidate because project 18d5cf0251234c518d493e8a529999d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:27:36.303017 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-6c1bf69b-8dec-4e34-8467-1eccb784bc64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:36.303017 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-6c1bf69b-8dec-4e34-8467-1eccb784bc64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:36.310534 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1048/2090] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:27:35 2022] DELETE /identity/v3/projects/18d5cf0251234c518d493e8a529999d0 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:36.311537 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1bf69b-8dec-4e34-8467-1eccb784bc64 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:36.314654 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-6c1bf69b-8dec-4e34-8467-1eccb784bc64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:36.412806 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1bf69b-8dec-4e34-8467-1eccb784bc64 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:36.413003 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dcfda5-a988-4ad6-aed8-05c8f3dcaa86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:36.413480 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dcfda5-a988-4ad6-aed8-05c8f3dcaa86 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:36.414648 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dcfda5-a988-4ad6-aed8-05c8f3dcaa86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:36.416028 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dcfda5-a988-4ad6-aed8-05c8f3dcaa86 admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:36.416699 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dcfda5-a988-4ad6-aed8-05c8f3dcaa86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:36.419532 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99a5ea-360c-434b-9a6e-12e6f6c9e90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:36.420398 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99a5ea-360c-434b-9a6e-12e6f6c9e90c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:36.420684 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99a5ea-360c-434b-9a6e-12e6f6c9e90c admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:36.420790 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dcfda5-a988-4ad6-aed8-05c8f3dcaa86 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:36.420959 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99a5ea-360c-434b-9a6e-12e6f6c9e90c admin admin] PATH_INFO: `/v3/users/cdc6fa57cc7c497b9224dad34bd355e4` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:36.422137 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb99a5ea-360c-434b-9a6e-12e6f6c9e90c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdc6fa57cc7c497b9224dad34bd355e4)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:36.424763 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb99a5ea-360c-434b-9a6e-12e6f6c9e90c admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:36.459068 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1043/2091] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:27:36 2022] POST /identity/v3/users => generated 304 bytes in 422 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:27:36.460249 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dcfda5-a988-4ad6-aed8-05c8f3dcaa86 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:36.462180 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-c8dcfda5-a988-4ad6-aed8-05c8f3dcaa86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:36.475433 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dcfda5-a988-4ad6-aed8-05c8f3dcaa86 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:36.480529 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2697ce1f-91d3-4c3a-9104-e22282da10b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:36.481533 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697ce1f-91d3-4c3a-9104-e22282da10b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:36.482131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697ce1f-91d3-4c3a-9104-e22282da10b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:36.482618 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697ce1f-91d3-4c3a-9104-e22282da10b7 admin admin] PATH_INFO: `/v3/users/5f01710a590343e4ab30bba3bda0ea42` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:36.483889 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2697ce1f-91d3-4c3a-9104-e22282da10b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f01710a590343e4ab30bba3bda0ea42)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:36.485437 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-fb99a5ea-360c-434b-9a6e-12e6f6c9e90c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdc6fa57cc7c497b9224dad34bd355e4'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:36.486296 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-fb99a5ea-360c-434b-9a6e-12e6f6c9e90c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdc6fa57cc7c497b9224dad34bd355e4'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:36.486406 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2697ce1f-91d3-4c3a-9104-e22282da10b7 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:36.487134 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-fb99a5ea-360c-434b-9a6e-12e6f6c9e90c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdc6fa57cc7c497b9224dad34bd355e4'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:36.504972 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-fb99a5ea-360c-434b-9a6e-12e6f6c9e90c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdc6fa57cc7c497b9224dad34bd355e4'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:36.514870 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-fb99a5ea-360c-434b-9a6e-12e6f6c9e90c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdc6fa57cc7c497b9224dad34bd355e4'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:36.536041 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1049/2092] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:27:36 2022] DELETE /identity/v3/users/cdc6fa57cc7c497b9224dad34bd355e4 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:36.536041 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99a5ea-360c-434b-9a6e-12e6f6c9e90c admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:36.536041 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-fb99a5ea-360c-434b-9a6e-12e6f6c9e90c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:36.551606 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-2697ce1f-91d3-4c3a-9104-e22282da10b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f01710a590343e4ab30bba3bda0ea42'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:36.552406 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-2697ce1f-91d3-4c3a-9104-e22282da10b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f01710a590343e4ab30bba3bda0ea42'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:36.561769 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=84,keystone:DELETE=12,keystone:INSERT=4 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:36.564360 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-2697ce1f-91d3-4c3a-9104-e22282da10b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f01710a590343e4ab30bba3bda0ea42'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:36.565524 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-2697ce1f-91d3-4c3a-9104-e22282da10b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f01710a590343e4ab30bba3bda0ea42'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:36.594421 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-2697ce1f-91d3-4c3a-9104-e22282da10b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f01710a590343e4ab30bba3bda0ea42'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:36.623278 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1044/2093] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:27:36 2022] DELETE /identity/v3/users/5f01710a590343e4ab30bba3bda0ea42 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:36.668303 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99a5ea-360c-434b-9a6e-12e6f6c9e90c admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:36.673876 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:36.759470 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c18d4-b30e-4840-a175-5874c34b4251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:36.759470 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c18d4-b30e-4840-a175-5874c34b4251 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:36.759470 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c18d4-b30e-4840-a175-5874c34b4251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:36.759470 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c18d4-b30e-4840-a175-5874c34b4251 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:36.759860 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8c18d4-b30e-4840-a175-5874c34b4251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:36.759860 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8c18d4-b30e-4840-a175-5874c34b4251 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:36.776032 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1050/2094] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:27:36.777150 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2697ce1f-91d3-4c3a-9104-e22282da10b7 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:36.778745 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-2697ce1f-91d3-4c3a-9104-e22282da10b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:36.795995 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2697ce1f-91d3-4c3a-9104-e22282da10b7 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:36.802896 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e98265-178c-4682-bd44-c47718878e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:36.803234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e98265-178c-4682-bd44-c47718878e4a admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:36.804165 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e98265-178c-4682-bd44-c47718878e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:36.804634 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e98265-178c-4682-bd44-c47718878e4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:36.805647 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e98265-178c-4682-bd44-c47718878e4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:36.808054 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e98265-178c-4682-bd44-c47718878e4a admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:36.839293 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c18d4-b30e-4840-a175-5874c34b4251 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:36.839478 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1045/2095] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:27:36.853446 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c18d4-b30e-4840-a175-5874c34b4251 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:36.858625 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca95d8-834f-49d6-8744-8d6e22b7a4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:36.860412 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca95d8-834f-49d6-8744-8d6e22b7a4dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:36.860706 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca95d8-834f-49d6-8744-8d6e22b7a4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:36.861085 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca95d8-834f-49d6-8744-8d6e22b7a4dc admin admin] PATH_INFO: `/v3/projects/f46ad7a1a22049b0a6bef212498844e9/users/06d4574818ee42abbb430981734518a5/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:36.881006 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aca95d8-834f-49d6-8744-8d6e22b7a4dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f46ad7a1a22049b0a6bef212498844e9, user_id=06d4574818ee42abbb430981734518a5, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:36.884448 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aca95d8-834f-49d6-8744-8d6e22b7a4dc admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:36.899836 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1051/2096] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:27:36 2022] PUT /identity/v3/projects/f46ad7a1a22049b0a6bef212498844e9/users/06d4574818ee42abbb430981734518a5/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:27:36.911989 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9623b6b-b5b4-48fa-8642-b76a9bf53db3 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:36.912350 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9623b6b-b5b4-48fa-8642-b76a9bf53db3 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:36.912708 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9623b6b-b5b4-48fa-8642-b76a9bf53db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:36.945729 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-c9623b6b-b5b4-48fa-8642-b76a9bf53db3 None None] MFA Rules not processed for user `06d4574818ee42abbb430981734518a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:36.947086 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-c9623b6b-b5b4-48fa-8642-b76a9bf53db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:37.035535 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1046/2097] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:36 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:27:37.041133 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca95d8-834f-49d6-8744-8d6e22b7a4dc admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:37.051994 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9623b6b-b5b4-48fa-8642-b76a9bf53db3 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:37.058204 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-c9623b6b-b5b4-48fa-8642-b76a9bf53db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:37.176889 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca95d8-834f-49d6-8744-8d6e22b7a4dc admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:37.181720 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bf70ce-d38b-44f7-9610-8ebe96364aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:37.182863 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf70ce-d38b-44f7-9610-8ebe96364aee admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:37.182863 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf70ce-d38b-44f7-9610-8ebe96364aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:37.183079 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf70ce-d38b-44f7-9610-8ebe96364aee admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:37.183656 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bf70ce-d38b-44f7-9610-8ebe96364aee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:37.185882 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bf70ce-d38b-44f7-9610-8ebe96364aee admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:37.197439 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9623b6b-b5b4-48fa-8642-b76a9bf53db3 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:37.203375 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18ecdb2-4ebd-4a61-a456-a9f15e298bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:37.206033 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ecdb2-4ebd-4a61-a456-a9f15e298bc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:37.206461 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ecdb2-4ebd-4a61-a456-a9f15e298bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:37.207329 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ecdb2-4ebd-4a61-a456-a9f15e298bc6 admin admin] PATH_INFO: `/v3/projects/da29f394f9a3424cabfb2344e894e5e5` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:37.208940 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18ecdb2-4ebd-4a61-a456-a9f15e298bc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da29f394f9a3424cabfb2344e894e5e5)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:37.211994 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18ecdb2-4ebd-4a61-a456-a9f15e298bc6 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:37.234615 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1052/2098] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:27:37 2022] POST /identity/v3/projects => generated 409 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:27:37.236429 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bf70ce-d38b-44f7-9610-8ebe96364aee admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:37.238708 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-b2bf70ce-d38b-44f7-9610-8ebe96364aee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:37.342911 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bf70ce-d38b-44f7-9610-8ebe96364aee admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:37.356974 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:37.359174 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-b18ecdb2-4ebd-4a61-a456-a9f15e298bc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da29f394f9a3424cabfb2344e894e5e5', 'request_id': 'req-b18ecdb2-4ebd-4a61-a456-a9f15e298bc6'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:37.361247 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:37.377841 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-b18ecdb2-4ebd-4a61-a456-a9f15e298bc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da29f394f9a3424cabfb2344e894e5e5', 'request_id': 'req-b18ecdb2-4ebd-4a61-a456-a9f15e298bc6'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:37.391263 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-b18ecdb2-4ebd-4a61-a456-a9f15e298bc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da29f394f9a3424cabfb2344e894e5e5', 'request_id': 'req-b18ecdb2-4ebd-4a61-a456-a9f15e298bc6'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:37.413923 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-b18ecdb2-4ebd-4a61-a456-a9f15e298bc6 admin admin] The token cache is being invalidate because project da29f394f9a3424cabfb2344e894e5e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:27:37.414297 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-b18ecdb2-4ebd-4a61-a456-a9f15e298bc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:37.415038 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-b18ecdb2-4ebd-4a61-a456-a9f15e298bc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:37.418917 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1047/2099] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:27:37 2022] DELETE /identity/v3/projects/da29f394f9a3424cabfb2344e894e5e5 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:37.420528 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18ecdb2-4ebd-4a61-a456-a9f15e298bc6 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:37.427619 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-b18ecdb2-4ebd-4a61-a456-a9f15e298bc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:37.450531 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1e9e8-417d-44fd-ad4c-07edc024429a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:37.451712 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1e9e8-417d-44fd-ad4c-07edc024429a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:37.452248 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1e9e8-417d-44fd-ad4c-07edc024429a admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:37.452946 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1e9e8-417d-44fd-ad4c-07edc024429a admin admin] PATH_INFO: `/v3/projects/f75af7d106f24083af134d63da0dcf3d` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:37.455037 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f1e9e8-417d-44fd-ad4c-07edc024429a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f75af7d106f24083af134d63da0dcf3d)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:37.458391 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f1e9e8-417d-44fd-ad4c-07edc024429a admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:37.512117 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18ecdb2-4ebd-4a61-a456-a9f15e298bc6 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:37.530351 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d3f1e9e8-417d-44fd-ad4c-07edc024429a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f75af7d106f24083af134d63da0dcf3d', 'request_id': 'req-d3f1e9e8-417d-44fd-ad4c-07edc024429a'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:37.537012 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d3f1e9e8-417d-44fd-ad4c-07edc024429a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f75af7d106f24083af134d63da0dcf3d', 'request_id': 'req-d3f1e9e8-417d-44fd-ad4c-07edc024429a'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:37.544027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d3f1e9e8-417d-44fd-ad4c-07edc024429a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f75af7d106f24083af134d63da0dcf3d', 'request_id': 'req-d3f1e9e8-417d-44fd-ad4c-07edc024429a'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:37.558281 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d3f1e9e8-417d-44fd-ad4c-07edc024429a admin admin] The token cache is being invalidate because project f75af7d106f24083af134d63da0dcf3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:27:37.558465 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d3f1e9e8-417d-44fd-ad4c-07edc024429a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:37.559374 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d3f1e9e8-417d-44fd-ad4c-07edc024429a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:37.561156 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1053/2100] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:27:37 2022] DELETE /identity/v3/projects/f75af7d106f24083af134d63da0dcf3d => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:37.562032 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1e9e8-417d-44fd-ad4c-07edc024429a admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:37.563484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-d3f1e9e8-417d-44fd-ad4c-07edc024429a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:37.592581 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791e4f2-3104-4578-84ee-2a12760182c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:37.593197 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791e4f2-3104-4578-84ee-2a12760182c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:37.593483 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791e4f2-3104-4578-84ee-2a12760182c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:37.593757 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791e4f2-3104-4578-84ee-2a12760182c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:37.594389 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5791e4f2-3104-4578-84ee-2a12760182c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:37.596469 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5791e4f2-3104-4578-84ee-2a12760182c4 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:37.624766 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1048/2101] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:27:37 2022] POST /identity/v3/users => generated 311 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:27:37.625563 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791e4f2-3104-4578-84ee-2a12760182c4 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:37.627525 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-5791e4f2-3104-4578-84ee-2a12760182c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:37.640183 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791e4f2-3104-4578-84ee-2a12760182c4 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:37.641857 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1e9e8-417d-44fd-ad4c-07edc024429a admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:37.645394 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c746f-9016-4a07-9209-0d05f75aa6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:37.646093 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c746f-9016-4a07-9209-0d05f75aa6ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:37.646439 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c746f-9016-4a07-9209-0d05f75aa6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:37.646778 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c746f-9016-4a07-9209-0d05f75aa6ab admin admin] PATH_INFO: `/v3/users/8d70cbabe0d34b60ab2935b3dde29b6b` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:37.648482 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85c746f-9016-4a07-9209-0d05f75aa6ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d70cbabe0d34b60ab2935b3dde29b6b)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:37.649337 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8510f22e-f223-4495-9700-6f937fcf28e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:37.650412 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85c746f-9016-4a07-9209-0d05f75aa6ab admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:37.650577 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8510f22e-f223-4495-9700-6f937fcf28e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:37.650889 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8510f22e-f223-4495-9700-6f937fcf28e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:37.651170 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8510f22e-f223-4495-9700-6f937fcf28e2 admin admin] PATH_INFO: `/v3/projects/efedff4d28bf468dbd796b113e134470` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:37.652679 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8510f22e-f223-4495-9700-6f937fcf28e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efedff4d28bf468dbd796b113e134470)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:37.657376 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8510f22e-f223-4495-9700-6f937fcf28e2 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:37.707073 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-f85c746f-9016-4a07-9209-0d05f75aa6ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d70cbabe0d34b60ab2935b3dde29b6b'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:37.707823 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-f85c746f-9016-4a07-9209-0d05f75aa6ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d70cbabe0d34b60ab2935b3dde29b6b'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:37.709100 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:37.714920 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-8510f22e-f223-4495-9700-6f937fcf28e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efedff4d28bf468dbd796b113e134470', 'request_id': 'req-8510f22e-f223-4495-9700-6f937fcf28e2'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:37.716998 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-f85c746f-9016-4a07-9209-0d05f75aa6ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d70cbabe0d34b60ab2935b3dde29b6b'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:37.717709 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-f85c746f-9016-4a07-9209-0d05f75aa6ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d70cbabe0d34b60ab2935b3dde29b6b'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:37.725451 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-8510f22e-f223-4495-9700-6f937fcf28e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efedff4d28bf468dbd796b113e134470', 'request_id': 'req-8510f22e-f223-4495-9700-6f937fcf28e2'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:37.729223 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:37.730615 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-f85c746f-9016-4a07-9209-0d05f75aa6ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d70cbabe0d34b60ab2935b3dde29b6b'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:37.736492 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-8510f22e-f223-4495-9700-6f937fcf28e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efedff4d28bf468dbd796b113e134470', 'request_id': 'req-8510f22e-f223-4495-9700-6f937fcf28e2'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:37.747493 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1049/2102] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:27:37 2022] DELETE /identity/v3/users/8d70cbabe0d34b60ab2935b3dde29b6b => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:37.750715 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65670da3-4169-422d-8b58-ec41e844ad6f None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:37.750715 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65670da3-4169-422d-8b58-ec41e844ad6f None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:37.750715 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65670da3-4169-422d-8b58-ec41e844ad6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:37.757415 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-8510f22e-f223-4495-9700-6f937fcf28e2 admin admin] The token cache is being invalidate because project efedff4d28bf468dbd796b113e134470 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:27:37.757726 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-8510f22e-f223-4495-9700-6f937fcf28e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:37.758474 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-8510f22e-f223-4495-9700-6f937fcf28e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:37.760521 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1054/2103] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:27:37 2022] DELETE /identity/v3/projects/efedff4d28bf468dbd796b113e134470 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:37.765838 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8510f22e-f223-4495-9700-6f937fcf28e2 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:37.767417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-8510f22e-f223-4495-9700-6f937fcf28e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:37.770349 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-65670da3-4169-422d-8b58-ec41e844ad6f None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:37.771677 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-65670da3-4169-422d-8b58-ec41e844ad6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:37.893548 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1050/2104] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:27:37.906184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f76b8-55e5-4625-ac18-24131bd9fc1b None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:37.906410 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f76b8-55e5-4625-ac18-24131bd9fc1b None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:37.908861 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f76b8-55e5-4625-ac18-24131bd9fc1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:37.950544 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8510f22e-f223-4495-9700-6f937fcf28e2 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:37.962015 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9a0fe4-eba2-4dec-a2e3-b73d80d0b4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:37.962650 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a0fe4-eba2-4dec-a2e3-b73d80d0b4e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:37.962976 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a0fe4-eba2-4dec-a2e3-b73d80d0b4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:37.963252 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a0fe4-eba2-4dec-a2e3-b73d80d0b4e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:37.963911 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9a0fe4-eba2-4dec-a2e3-b73d80d0b4e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:37.965514 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9a0fe4-eba2-4dec-a2e3-b73d80d0b4e0 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:37.967364 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-340f76b8-55e5-4625-ac18-24131bd9fc1b None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:37.968663 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-340f76b8-55e5-4625-ac18-24131bd9fc1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:37.984883 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1055/2105] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:27:37.987541 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41c291-be8f-4f16-83f0-dfa94780a969 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:37.987836 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41c291-be8f-4f16-83f0-dfa94780a969 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:37.988340 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41c291-be8f-4f16-83f0-dfa94780a969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:37.993942 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1051/2106] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:27:37.994297 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340f76b8-55e5-4625-ac18-24131bd9fc1b None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:38.013703 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340f76b8-55e5-4625-ac18-24131bd9fc1b None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:38.029056 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c22e6b-2e2b-4f41-a235-f06d14568841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:38.029856 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c22e6b-2e2b-4f41-a235-f06d14568841 admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:38.030008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c22e6b-2e2b-4f41-a235-f06d14568841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:38.030284 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c22e6b-2e2b-4f41-a235-f06d14568841 admin admin] PATH_INFO: `/v3/projects/ca8bd83bab374e708b66af809461c326/users/01f08201de2944d0a79ebc8de0d13eb9/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:38.036455 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-3e41c291-be8f-4f16-83f0-dfa94780a969 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:38.040775 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-3e41c291-be8f-4f16-83f0-dfa94780a969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:38.059363 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1056/2107] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:27:38.060382 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf145533-39cc-453b-b548-53c1418cf65b None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:38.060667 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf145533-39cc-453b-b548-53c1418cf65b None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:38.060941 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf145533-39cc-453b-b548-53c1418cf65b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:38.062118 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c22e6b-2e2b-4f41-a235-f06d14568841 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca8bd83bab374e708b66af809461c326, user_id=01f08201de2944d0a79ebc8de0d13eb9, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:38.065169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c22e6b-2e2b-4f41-a235-f06d14568841 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:38.086882 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1052/2108] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:27:37 2022] PUT /identity/v3/projects/ca8bd83bab374e708b66af809461c326/users/01f08201de2944d0a79ebc8de0d13eb9/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:38.087811 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c22e6b-2e2b-4f41-a235-f06d14568841 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:38.096131 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-cf145533-39cc-453b-b548-53c1418cf65b None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:38.098013 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-cf145533-39cc-453b-b548-53c1418cf65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:38.169997 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c22e6b-2e2b-4f41-a235-f06d14568841 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:38.175362 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73770170-a47e-4835-918f-19e441d5340f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:38.175965 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73770170-a47e-4835-918f-19e441d5340f admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:38.176720 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73770170-a47e-4835-918f-19e441d5340f admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:38.176720 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73770170-a47e-4835-918f-19e441d5340f admin admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:38.177132 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73770170-a47e-4835-918f-19e441d5340f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:27:38.177430 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73770170-a47e-4835-918f-19e441d5340f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:38.179835 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73770170-a47e-4835-918f-19e441d5340f admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:38.185614 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1057/2109] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:27:38.187686 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf145533-39cc-453b-b548-53c1418cf65b None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:38.196702 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1053/2110] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:27:38.198005 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73770170-a47e-4835-918f-19e441d5340f admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:38.199808 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-73770170-a47e-4835-918f-19e441d5340f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:38.210245 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf145533-39cc-453b-b548-53c1418cf65b None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:38.215160 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f863d-07d3-441d-9d73-6bbe17df49af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:38.216054 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f863d-07d3-441d-9d73-6bbe17df49af admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:38.216337 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f863d-07d3-441d-9d73-6bbe17df49af admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:38.216643 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73770170-a47e-4835-918f-19e441d5340f admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:38.216812 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f863d-07d3-441d-9d73-6bbe17df49af admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:38.217730 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f863d-07d3-441d-9d73-6bbe17df49af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:38.221095 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f863d-07d3-441d-9d73-6bbe17df49af admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:38.221873 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ce018-a421-41d2-8237-727a0e0a08b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:38.222638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ce018-a421-41d2-8237-727a0e0a08b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:38.222959 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ce018-a421-41d2-8237-727a0e0a08b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:38.223237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ce018-a421-41d2-8237-727a0e0a08b1 admin admin] PATH_INFO: `/v3/projects/bad95c06cd95470f8b783a2bd984209e` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:38.224412 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ce018-a421-41d2-8237-727a0e0a08b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bad95c06cd95470f8b783a2bd984209e)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:38.227271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ce018-a421-41d2-8237-727a0e0a08b1 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:38.239226 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1058/2111] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:27:38.239962 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f863d-07d3-441d-9d73-6bbe17df49af admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:38.251972 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f863d-07d3-441d-9d73-6bbe17df49af admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:38.258141 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb391b8-2d23-481c-8c6a-54279ce003ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:38.258141 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb391b8-2d23-481c-8c6a-54279ce003ae admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:38.258141 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb391b8-2d23-481c-8c6a-54279ce003ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:38.258868 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb391b8-2d23-481c-8c6a-54279ce003ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:38.259416 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb391b8-2d23-481c-8c6a-54279ce003ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:27:38.259911 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb391b8-2d23-481c-8c6a-54279ce003ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:38.261688 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb391b8-2d23-481c-8c6a-54279ce003ae admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:38.275779 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1059/2112] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:27:38.276890 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb391b8-2d23-481c-8c6a-54279ce003ae admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:38.280280 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:38.285403 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-431ce018-a421-41d2-8237-727a0e0a08b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bad95c06cd95470f8b783a2bd984209e', 'request_id': 'req-431ce018-a421-41d2-8237-727a0e0a08b1'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:38.287578 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb391b8-2d23-481c-8c6a-54279ce003ae admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:38.296703 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-431ce018-a421-41d2-8237-727a0e0a08b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bad95c06cd95470f8b783a2bd984209e', 'request_id': 'req-431ce018-a421-41d2-8237-727a0e0a08b1'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:38.305033 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-431ce018-a421-41d2-8237-727a0e0a08b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bad95c06cd95470f8b783a2bd984209e', 'request_id': 'req-431ce018-a421-41d2-8237-727a0e0a08b1'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:38.324150 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-431ce018-a421-41d2-8237-727a0e0a08b1 admin admin] The token cache is being invalidate because project bad95c06cd95470f8b783a2bd984209e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:27:38.325240 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-431ce018-a421-41d2-8237-727a0e0a08b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:38.325240 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-431ce018-a421-41d2-8237-727a0e0a08b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:38.326680 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1054/2113] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:27:38 2022] DELETE /identity/v3/projects/bad95c06cd95470f8b783a2bd984209e => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:38.330674 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:38.330867 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ce018-a421-41d2-8237-727a0e0a08b1 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:38.332400 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-431ce018-a421-41d2-8237-727a0e0a08b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:38.385867 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba67c777-c714-498c-8de4-d112371ae457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:38.386101 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67c777-c714-498c-8de4-d112371ae457 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:38.386386 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67c777-c714-498c-8de4-d112371ae457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:38.386695 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67c777-c714-498c-8de4-d112371ae457 admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:38.387323 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba67c777-c714-498c-8de4-d112371ae457 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:38.389897 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba67c777-c714-498c-8de4-d112371ae457 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:38.412239 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ce018-a421-41d2-8237-727a0e0a08b1 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:38.418571 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d4395-768a-44a5-8498-a48c66edf859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:38.419221 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d4395-768a-44a5-8498-a48c66edf859 admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:38.419583 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d4395-768a-44a5-8498-a48c66edf859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:38.419981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d4395-768a-44a5-8498-a48c66edf859 admin admin] PATH_INFO: `/v3/projects/ca8bd83bab374e708b66af809461c326/users/01f08201de2944d0a79ebc8de0d13eb9/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:38.422198 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4d4395-768a-44a5-8498-a48c66edf859 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca8bd83bab374e708b66af809461c326, user_id=01f08201de2944d0a79ebc8de0d13eb9, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:38.423071 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1060/2114] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:27:38 2022] POST /identity/v3/projects => generated 375 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:27:38.424100 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba67c777-c714-498c-8de4-d112371ae457 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:38.425420 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4d4395-768a-44a5-8498-a48c66edf859 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:38.426009 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-ba67c777-c714-498c-8de4-d112371ae457 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:38.438867 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1055/2115] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:27:38 2022] PUT /identity/v3/projects/ca8bd83bab374e708b66af809461c326/users/01f08201de2944d0a79ebc8de0d13eb9/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:38.440260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327753a-54cc-48dd-872e-795188b8ab73 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:38.440613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327753a-54cc-48dd-872e-795188b8ab73 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:38.440955 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327753a-54cc-48dd-872e-795188b8ab73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:38.481502 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-f327753a-54cc-48dd-872e-795188b8ab73 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:38.482853 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-f327753a-54cc-48dd-872e-795188b8ab73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:38.554991 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba67c777-c714-498c-8de4-d112371ae457 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:38.598332 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1056/2116] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:27:38.599661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f327753a-54cc-48dd-872e-795188b8ab73 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:38.601906 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-f327753a-54cc-48dd-872e-795188b8ab73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:38.616660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f327753a-54cc-48dd-872e-795188b8ab73 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:38.623150 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05678b6d-40be-4bb3-888b-84e3644a5262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:38.623768 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05678b6d-40be-4bb3-888b-84e3644a5262 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:38.624042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05678b6d-40be-4bb3-888b-84e3644a5262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:38.624309 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05678b6d-40be-4bb3-888b-84e3644a5262 admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:38.624945 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05678b6d-40be-4bb3-888b-84e3644a5262 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:38.626889 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05678b6d-40be-4bb3-888b-84e3644a5262 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:38.655604 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1057/2117] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:27:38 2022] POST /identity/v3/users => generated 288 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:27:38.656444 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde2c93-427a-4acc-9715-83f59ca6ef05 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:38.656728 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde2c93-427a-4acc-9715-83f59ca6ef05 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:38.657019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde2c93-427a-4acc-9715-83f59ca6ef05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:38.661142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7284d3-5511-4322-b183-9be4d802cd93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:38.662591 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7284d3-5511-4322-b183-9be4d802cd93 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:38.664246 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7284d3-5511-4322-b183-9be4d802cd93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:38.667551 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7284d3-5511-4322-b183-9be4d802cd93 admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:38.668711 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7284d3-5511-4322-b183-9be4d802cd93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:38.672430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7284d3-5511-4322-b183-9be4d802cd93 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:38.700679 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-6fde2c93-427a-4acc-9715-83f59ca6ef05 None None] MFA Rules not processed for user `01f08201de2944d0a79ebc8de0d13eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:38.702045 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-6fde2c93-427a-4acc-9715-83f59ca6ef05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:38.722430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1061/2118] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:27:38 2022] POST /identity/v3/projects => generated 387 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:27:38.728250 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177936e-3c7b-4c4b-8bb1-a21764a1a2af None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:38.728557 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177936e-3c7b-4c4b-8bb1-a21764a1a2af None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:38.728821 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177936e-3c7b-4c4b-8bb1-a21764a1a2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:38.751039 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-2177936e-3c7b-4c4b-8bb1-a21764a1a2af None None] MFA Rules not processed for user `d381a365e0744834b716fa6b0ea18d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:38.752409 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-2177936e-3c7b-4c4b-8bb1-a21764a1a2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:38.761735 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:38.810145 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1058/2119] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:38 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:27:38.811260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56853fca-eed2-46fc-9fb7-d056e7e488de None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:38.811546 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56853fca-eed2-46fc-9fb7-d056e7e488de None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:38.811840 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56853fca-eed2-46fc-9fb7-d056e7e488de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:38.814066 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:38.835692 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1062/2120] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:27:38.836792 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177936e-3c7b-4c4b-8bb1-a21764a1a2af None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:38.841078 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-56853fca-eed2-46fc-9fb7-d056e7e488de None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:38.842290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-56853fca-eed2-46fc-9fb7-d056e7e488de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:38.947475 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177936e-3c7b-4c4b-8bb1-a21764a1a2af None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:38.950593 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1059/2121] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:27:38.951616 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56853fca-eed2-46fc-9fb7-d056e7e488de None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:38.959022 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be65b59-2e94-47d4-8e91-7070ce1aba3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:38.960855 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be65b59-2e94-47d4-8e91-7070ce1aba3d admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:38.961136 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be65b59-2e94-47d4-8e91-7070ce1aba3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:38.961410 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be65b59-2e94-47d4-8e91-7070ce1aba3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:38.962563 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be65b59-2e94-47d4-8e91-7070ce1aba3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:38.964348 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be65b59-2e94-47d4-8e91-7070ce1aba3d admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:38.973086 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56853fca-eed2-46fc-9fb7-d056e7e488de None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:38.981707 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1063/2122] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:27:38.981707 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be65b59-2e94-47d4-8e91-7070ce1aba3d admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:38.983231 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-9be65b59-2e94-47d4-8e91-7070ce1aba3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:38.992405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e2bf10-5554-43ed-90a1-7be161b66637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:38.993180 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e2bf10-5554-43ed-90a1-7be161b66637 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:38.993405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e2bf10-5554-43ed-90a1-7be161b66637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:38.993644 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e2bf10-5554-43ed-90a1-7be161b66637 admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:38.994297 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e2bf10-5554-43ed-90a1-7be161b66637 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:38.997583 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e2bf10-5554-43ed-90a1-7be161b66637 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:39.030550 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1060/2123] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:27:38 2022] POST /identity/v3/users => generated 294 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:27:39.032641 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e2bf10-5554-43ed-90a1-7be161b66637 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:39.049347 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e2bf10-5554-43ed-90a1-7be161b66637 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:39.060864 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d77710d-b449-4525-af2f-2d67efe655dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:39.061820 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77710d-b449-4525-af2f-2d67efe655dd admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:39.062772 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77710d-b449-4525-af2f-2d67efe655dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:39.063243 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77710d-b449-4525-af2f-2d67efe655dd admin admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:39.064029 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d77710d-b449-4525-af2f-2d67efe655dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:27:39.064562 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d77710d-b449-4525-af2f-2d67efe655dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:39.066397 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d77710d-b449-4525-af2f-2d67efe655dd admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:39.085902 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1061/2124] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:27:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:27:39.086869 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d77710d-b449-4525-af2f-2d67efe655dd admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:39.093452 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be65b59-2e94-47d4-8e91-7070ce1aba3d admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:39.097640 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c6d58-a6fb-445c-822a-111473fbed8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:39.098259 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c6d58-a6fb-445c-822a-111473fbed8e service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:39.098536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c6d58-a6fb-445c-822a-111473fbed8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:39.098803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c6d58-a6fb-445c-822a-111473fbed8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:39.101396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d77710d-b449-4525-af2f-2d67efe655dd admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:39.107682 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42cd236-3481-4df1-aef8-7c6bc9b17b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:39.108794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cd236-3481-4df1-aef8-7c6bc9b17b57 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:39.115316 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cd236-3481-4df1-aef8-7c6bc9b17b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:39.115633 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cd236-3481-4df1-aef8-7c6bc9b17b57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:39.116860 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42cd236-3481-4df1-aef8-7c6bc9b17b57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:39.124188 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42cd236-3481-4df1-aef8-7c6bc9b17b57 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:39.143258 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1062/2125] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:27:39.143258 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42cd236-3481-4df1-aef8-7c6bc9b17b57 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:39.153707 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42cd236-3481-4df1-aef8-7c6bc9b17b57 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:39.166856 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6867c0-3e21-47e8-85b7-5091641c32a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:39.168007 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6867c0-3e21-47e8-85b7-5091641c32a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:39.168007 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6867c0-3e21-47e8-85b7-5091641c32a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:39.168007 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6867c0-3e21-47e8-85b7-5091641c32a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:39.168404 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6867c0-3e21-47e8-85b7-5091641c32a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:39.170013 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6867c0-3e21-47e8-85b7-5091641c32a9 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:39.184628 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1063/2126] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:27:39.184628 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6867c0-3e21-47e8-85b7-5091641c32a9 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:39.198778 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6867c0-3e21-47e8-85b7-5091641c32a9 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:39.207793 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c81916-47ba-4aa4-85e2-6057ae635242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:39.209261 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c81916-47ba-4aa4-85e2-6057ae635242 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:39.210576 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c81916-47ba-4aa4-85e2-6057ae635242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:39.211607 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c81916-47ba-4aa4-85e2-6057ae635242 admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:39.212781 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c81916-47ba-4aa4-85e2-6057ae635242 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:39.215265 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c81916-47ba-4aa4-85e2-6057ae635242 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:39.232427 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407c6d58-a6fb-445c-822a-111473fbed8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:39.234016 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407c6d58-a6fb-445c-822a-111473fbed8e service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:39.255340 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1064/2127] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:27:38 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:27:39.256194 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c6d58-a6fb-445c-822a-111473fbed8e service cinder] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:39.275871 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c6d58-a6fb-445c-822a-111473fbed8e service cinder] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:39.302496 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffce92c-862e-4104-8a33-f8467f73a07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:39.302496 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffce92c-862e-4104-8a33-f8467f73a07c admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:39.302496 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffce92c-862e-4104-8a33-f8467f73a07c admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:39.302496 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffce92c-862e-4104-8a33-f8467f73a07c admin admin] PATH_INFO: `/v3/projects/e19d1bcc7eca4c39842e754b7590e571/users/81dbf0922d4d4b62aa6c63fd92396c72/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:39.309716 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1064/2128] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:27:39 2022] POST /identity/v3/projects => generated 369 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:27:39.310684 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c81916-47ba-4aa4-85e2-6057ae635242 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:39.323225 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffce92c-862e-4104-8a33-f8467f73a07c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e19d1bcc7eca4c39842e754b7590e571, user_id=81dbf0922d4d4b62aa6c63fd92396c72, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:39.326949 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffce92c-862e-4104-8a33-f8467f73a07c admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:39.352089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1065/2129] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:27:39 2022] PUT /identity/v3/projects/e19d1bcc7eca4c39842e754b7590e571/users/81dbf0922d4d4b62aa6c63fd92396c72/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:39.352951 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffce92c-862e-4104-8a33-f8467f73a07c admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:39.410770 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:39.430188 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c81916-47ba-4aa4-85e2-6057ae635242 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:39.435701 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:39.478195 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffce92c-862e-4104-8a33-f8467f73a07c admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:39.483977 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998607b3-46dc-4383-9dd6-c04b03ebb5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:39.484943 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998607b3-46dc-4383-9dd6-c04b03ebb5f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:39.485520 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998607b3-46dc-4383-9dd6-c04b03ebb5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:39.485993 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998607b3-46dc-4383-9dd6-c04b03ebb5f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:39.487245 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998607b3-46dc-4383-9dd6-c04b03ebb5f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:39.489736 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998607b3-46dc-4383-9dd6-c04b03ebb5f9 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:39.520144 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1066/2130] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:27:39 2022] POST /identity/v3/users => generated 285 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:27:39.528643 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aefee9c-9c81-45ee-8f5c-2ea1e44ebf77 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:39.529050 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aefee9c-9c81-45ee-8f5c-2ea1e44ebf77 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:39.529328 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aefee9c-9c81-45ee-8f5c-2ea1e44ebf77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:39.529937 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa00a47a-3cfa-47ed-bc87-520c1ec3853a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:39.530894 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00a47a-3cfa-47ed-bc87-520c1ec3853a admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:39.531476 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00a47a-3cfa-47ed-bc87-520c1ec3853a admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:39.531927 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00a47a-3cfa-47ed-bc87-520c1ec3853a admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:39.541919 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa00a47a-3cfa-47ed-bc87-520c1ec3853a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:39.543923 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa00a47a-3cfa-47ed-bc87-520c1ec3853a admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:39.562378 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1065/2131] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:27:39.563469 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa00a47a-3cfa-47ed-bc87-520c1ec3853a admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:39.583522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa00a47a-3cfa-47ed-bc87-520c1ec3853a admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:39.588691 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce6bde-6a60-45c0-8c03-5352945e136b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:39.589375 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce6bde-6a60-45c0-8c03-5352945e136b admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:39.589658 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce6bde-6a60-45c0-8c03-5352945e136b admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:39.589923 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce6bde-6a60-45c0-8c03-5352945e136b admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:39.590672 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ce6bde-6a60-45c0-8c03-5352945e136b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:39.593100 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ce6bde-6a60-45c0-8c03-5352945e136b admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:39.598225 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-2aefee9c-9c81-45ee-8f5c-2ea1e44ebf77 None None] MFA Rules not processed for user `81dbf0922d4d4b62aa6c63fd92396c72`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:39.607651 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1066/2132] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:27:39.608801 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce6bde-6a60-45c0-8c03-5352945e136b admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:39.612527 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-2aefee9c-9c81-45ee-8f5c-2ea1e44ebf77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:39.687285 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce6bde-6a60-45c0-8c03-5352945e136b admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:39.691250 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d1fed-10e7-465d-a56a-225d448a961c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:39.691869 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d1fed-10e7-465d-a56a-225d448a961c service cinder] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:39.692143 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d1fed-10e7-465d-a56a-225d448a961c service cinder] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:39.693289 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d1fed-10e7-465d-a56a-225d448a961c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:39.695453 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-d86d1fed-10e7-465d-a56a-225d448a961c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:39.706740 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1067/2133] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:27:39.707900 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aefee9c-9c81-45ee-8f5c-2ea1e44ebf77 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:39.720632 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aefee9c-9c81-45ee-8f5c-2ea1e44ebf77 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:39.725945 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ace52-ca58-408f-accd-0dee2124134d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:39.726611 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ace52-ca58-408f-accd-0dee2124134d admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:39.727292 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ace52-ca58-408f-accd-0dee2124134d admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:39.727292 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ace52-ca58-408f-accd-0dee2124134d admin admin] PATH_INFO: `/v3/projects/ad9c3603c7224fdc997699d08317cf21/users/286d3cc6b18a49419cd9174d15a923e0/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:39.745817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ace52-ca58-408f-accd-0dee2124134d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad9c3603c7224fdc997699d08317cf21, user_id=286d3cc6b18a49419cd9174d15a923e0, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:39.748363 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ace52-ca58-408f-accd-0dee2124134d admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:39.774226 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1068/2134] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:27:39 2022] PUT /identity/v3/projects/ad9c3603c7224fdc997699d08317cf21/users/286d3cc6b18a49419cd9174d15a923e0/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:39.774462 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86d1fed-10e7-465d-a56a-225d448a961c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:39.774583 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ace52-ca58-408f-accd-0dee2124134d admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:39.774674 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86d1fed-10e7-465d-a56a-225d448a961c service cinder] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:39.781054 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1067/2135] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:27:39 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:27:39.781968 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d1fed-10e7-465d-a56a-225d448a961c service cinder] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:39.793336 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d1fed-10e7-465d-a56a-225d448a961c service cinder] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:39.874002 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:39.874341 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ace52-ca58-408f-accd-0dee2124134d admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:39.881009 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c1baa-a871-4307-a60e-4c76bfabafd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:39.881802 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c1baa-a871-4307-a60e-4c76bfabafd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:39.881941 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c1baa-a871-4307-a60e-4c76bfabafd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:39.882693 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c1baa-a871-4307-a60e-4c76bfabafd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:39.882803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7c1baa-a871-4307-a60e-4c76bfabafd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:39.884762 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7c1baa-a871-4307-a60e-4c76bfabafd0 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:39.896469 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1069/2136] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:27:39.901029 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57de59c-f62f-46c1-9ecb-107408dabb22 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:39.901890 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57de59c-f62f-46c1-9ecb-107408dabb22 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:39.901890 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57de59c-f62f-46c1-9ecb-107408dabb22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:39.903554 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b69a4-e290-41b3-9d4c-28ba8a6f8ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:39.904768 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b69a4-e290-41b3-9d4c-28ba8a6f8ded admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:39.905628 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b69a4-e290-41b3-9d4c-28ba8a6f8ded admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:39.906194 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b69a4-e290-41b3-9d4c-28ba8a6f8ded admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:39.908413 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6b69a4-e290-41b3-9d4c-28ba8a6f8ded admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:39.910420 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6b69a4-e290-41b3-9d4c-28ba8a6f8ded admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:39.929452 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:39.934901 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-e57de59c-f62f-46c1-9ecb-107408dabb22 None None] MFA Rules not processed for user `286d3cc6b18a49419cd9174d15a923e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:39.938721 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-e57de59c-f62f-46c1-9ecb-107408dabb22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:39.949833 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1068/2137] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:27:39 2022] POST /identity/v3/projects => generated 375 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:27:39.950762 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b69a4-e290-41b3-9d4c-28ba8a6f8ded admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:40.025239 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b69a4-e290-41b3-9d4c-28ba8a6f8ded admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:40.030905 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa88223-4240-49b5-8064-9fd3b0fbb205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:40.031989 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa88223-4240-49b5-8064-9fd3b0fbb205 admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:40.031989 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa88223-4240-49b5-8064-9fd3b0fbb205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:40.032564 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa88223-4240-49b5-8064-9fd3b0fbb205 admin admin] PATH_INFO: `/v3/projects/7eaa22c284634760bed017db5a3501c2/users/72bd93f257bf46d29af832f7691dd708/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:40.044918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1070/2138] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:39 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:27:40.045812 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57de59c-f62f-46c1-9ecb-107408dabb22 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:40.057949 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa88223-4240-49b5-8064-9fd3b0fbb205 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7eaa22c284634760bed017db5a3501c2, user_id=72bd93f257bf46d29af832f7691dd708, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:40.058554 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57de59c-f62f-46c1-9ecb-107408dabb22 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:40.061809 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa88223-4240-49b5-8064-9fd3b0fbb205 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:40.068613 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e4a06-102d-4772-8a95-02266a1f60b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:40.069784 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e4a06-102d-4772-8a95-02266a1f60b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:40.070123 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e4a06-102d-4772-8a95-02266a1f60b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:40.070459 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e4a06-102d-4772-8a95-02266a1f60b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:40.071182 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e4a06-102d-4772-8a95-02266a1f60b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:40.073677 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e4a06-102d-4772-8a95-02266a1f60b3 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:40.083935 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1069/2139] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:27:39 2022] PUT /identity/v3/projects/7eaa22c284634760bed017db5a3501c2/users/72bd93f257bf46d29af832f7691dd708/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:40.095204 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa88223-4240-49b5-8064-9fd3b0fbb205 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:40.109641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1071/2140] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:27:40 2022] POST /identity/v3/users => generated 294 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:27:40.113681 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19efb10a-8532-483b-94ac-fd8a905e4b38 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:40.113973 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19efb10a-8532-483b-94ac-fd8a905e4b38 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:40.114636 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19efb10a-8532-483b-94ac-fd8a905e4b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:40.175025 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa88223-4240-49b5-8064-9fd3b0fbb205 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:40.178483 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-19efb10a-8532-483b-94ac-fd8a905e4b38 None None] MFA Rules not processed for user `72bd93f257bf46d29af832f7691dd708`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:40.181472 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-19efb10a-8532-483b-94ac-fd8a905e4b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:40.202320 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c725c44-fe04-405f-ba34-858475411885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:40.202974 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c725c44-fe04-405f-ba34-858475411885 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:40.203256 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c725c44-fe04-405f-ba34-858475411885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:40.203527 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c725c44-fe04-405f-ba34-858475411885 admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:40.204162 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c725c44-fe04-405f-ba34-858475411885 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:40.206062 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c725c44-fe04-405f-ba34-858475411885 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:40.263903 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1070/2141] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:27:40 2022] POST /identity/v3/projects => generated 387 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:27:40.265128 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c725c44-fe04-405f-ba34-858475411885 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:40.381283 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:40.397647 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1072/2142] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:27:40.399535 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19efb10a-8532-483b-94ac-fd8a905e4b38 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:40.419564 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19efb10a-8532-483b-94ac-fd8a905e4b38 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:40.424318 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c725c44-fe04-405f-ba34-858475411885 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:40.426488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0daf5-ce99-4e84-8c8f-487329dd7d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:40.428224 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0daf5-ce99-4e84-8c8f-487329dd7d4b admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:40.428816 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0daf5-ce99-4e84-8c8f-487329dd7d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:40.429598 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0daf5-ce99-4e84-8c8f-487329dd7d4b admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:40.431588 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0daf5-ce99-4e84-8c8f-487329dd7d4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:40.431925 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de5146-743f-490a-8f44-4b7830b5b6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:40.432548 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de5146-743f-490a-8f44-4b7830b5b6da admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:40.432833 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de5146-743f-490a-8f44-4b7830b5b6da admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:40.433203 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de5146-743f-490a-8f44-4b7830b5b6da admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:40.434003 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de5146-743f-490a-8f44-4b7830b5b6da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:40.436180 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0daf5-ce99-4e84-8c8f-487329dd7d4b admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:40.436302 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de5146-743f-490a-8f44-4b7830b5b6da admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:40.458431 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1071/2143] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:27:40.460881 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de5146-743f-490a-8f44-4b7830b5b6da admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:40.466179 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-95de5146-743f-490a-8f44-4b7830b5b6da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:40.471789 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1073/2144] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:27:40 2022] POST /identity/v3/users => generated 294 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:27:40.472685 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0daf5-ce99-4e84-8c8f-487329dd7d4b admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:40.494713 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0daf5-ce99-4e84-8c8f-487329dd7d4b admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:40.507994 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072714b0-9259-46f7-8e26-c02680219963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:40.508694 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072714b0-9259-46f7-8e26-c02680219963 admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:40.515906 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072714b0-9259-46f7-8e26-c02680219963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:40.516776 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072714b0-9259-46f7-8e26-c02680219963 admin admin] PATH_INFO: `/v3/projects/f9a13890743242edae5996e02b625dde/users/71b66eb47d2f4b2c909215d72a9c9f3c/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:40.544495 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072714b0-9259-46f7-8e26-c02680219963 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9a13890743242edae5996e02b625dde, user_id=71b66eb47d2f4b2c909215d72a9c9f3c, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:40.547871 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072714b0-9259-46f7-8e26-c02680219963 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:40.572852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1074/2145] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:27:40 2022] PUT /identity/v3/projects/f9a13890743242edae5996e02b625dde/users/71b66eb47d2f4b2c909215d72a9c9f3c/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:40.575766 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072714b0-9259-46f7-8e26-c02680219963 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:40.626707 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:40.649502 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de5146-743f-490a-8f44-4b7830b5b6da admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:40.677928 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072714b0-9259-46f7-8e26-c02680219963 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:40.685167 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747b0d40-6002-4407-a049-728774bb6a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:40.687427 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747b0d40-6002-4407-a049-728774bb6a2e admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:40.687427 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747b0d40-6002-4407-a049-728774bb6a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:40.687427 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747b0d40-6002-4407-a049-728774bb6a2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:40.688084 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747b0d40-6002-4407-a049-728774bb6a2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:40.690088 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747b0d40-6002-4407-a049-728774bb6a2e admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:40.704429 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1075/2146] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:27:40.704429 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747b0d40-6002-4407-a049-728774bb6a2e admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:40.715543 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747b0d40-6002-4407-a049-728774bb6a2e admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:40.721189 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed248f-f522-4b55-938a-3e2e36e873bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:40.722509 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed248f-f522-4b55-938a-3e2e36e873bd admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:40.723129 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed248f-f522-4b55-938a-3e2e36e873bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:40.724112 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b8bc2-a570-4eaf-b450-f397e5521678 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e82b01dbd6e84a9d82baa2c28d77fc28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:40.725948 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b8bc2-a570-4eaf-b450-f397e5521678 service glance] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:40.726801 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b8bc2-a570-4eaf-b450-f397e5521678 service glance] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:40.726801 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b8bc2-a570-4eaf-b450-f397e5521678 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:40.733368 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed248f-f522-4b55-938a-3e2e36e873bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:40.735586 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ed248f-f522-4b55-938a-3e2e36e873bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:40.740668 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ed248f-f522-4b55-938a-3e2e36e873bd admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:40.757769 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1076/2147] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:27:40.758745 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed248f-f522-4b55-938a-3e2e36e873bd admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:40.771162 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed248f-f522-4b55-938a-3e2e36e873bd admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:40.776954 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09589051-c995-4493-9e7c-eca765442678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:40.777359 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09589051-c995-4493-9e7c-eca765442678 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:40.777636 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09589051-c995-4493-9e7c-eca765442678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:40.777911 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09589051-c995-4493-9e7c-eca765442678 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:40.779269 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09589051-c995-4493-9e7c-eca765442678 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:40.781354 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09589051-c995-4493-9e7c-eca765442678 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:40.794005 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1077/2148] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:27:40.795356 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09589051-c995-4493-9e7c-eca765442678 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:40.806855 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09589051-c995-4493-9e7c-eca765442678 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:40.812427 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86790094-2f65-43db-9916-5522cd092500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:40.813316 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86790094-2f65-43db-9916-5522cd092500 admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:40.813705 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86790094-2f65-43db-9916-5522cd092500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:40.813979 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86790094-2f65-43db-9916-5522cd092500 admin admin] PATH_INFO: `/v3/projects/f9a13890743242edae5996e02b625dde/users/71b66eb47d2f4b2c909215d72a9c9f3c/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:40.816298 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975b8bc2-a570-4eaf-b450-f397e5521678 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:40.818111 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975b8bc2-a570-4eaf-b450-f397e5521678 service glance] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:40.823270 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86790094-2f65-43db-9916-5522cd092500 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9a13890743242edae5996e02b625dde, user_id=71b66eb47d2f4b2c909215d72a9c9f3c, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:40.826359 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86790094-2f65-43db-9916-5522cd092500 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:40.829814 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1072/2149] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:27:40 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:27:40.830708 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b8bc2-a570-4eaf-b450-f397e5521678 service glance] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:40.843041 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b8bc2-a570-4eaf-b450-f397e5521678 service glance] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:40.849016 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1078/2150] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:27:40 2022] PUT /identity/v3/projects/f9a13890743242edae5996e02b625dde/users/71b66eb47d2f4b2c909215d72a9c9f3c/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:40.861200 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764fed2c-6739-4177-b137-64302e0c8970 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:40.861542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764fed2c-6739-4177-b137-64302e0c8970 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:40.861761 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764fed2c-6739-4177-b137-64302e0c8970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:40.878365 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:40.902027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-764fed2c-6739-4177-b137-64302e0c8970 None None] MFA Rules not processed for user `71b66eb47d2f4b2c909215d72a9c9f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:40.903310 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-764fed2c-6739-4177-b137-64302e0c8970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:40.924242 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1b257b-58b6-4bec-8786-b7c5ef6f2368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:40.924861 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b257b-58b6-4bec-8786-b7c5ef6f2368 admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:40.925497 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b257b-58b6-4bec-8786-b7c5ef6f2368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:40.925911 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b257b-58b6-4bec-8786-b7c5ef6f2368 admin admin] PATH_INFO: `/v3/projects/3f37e42e01664a5eb97e7b3f52b8c6f9/users/43402622442e436a9b19ca130b8d91ce/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:40.947696 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1b257b-58b6-4bec-8786-b7c5ef6f2368 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f37e42e01664a5eb97e7b3f52b8c6f9, user_id=43402622442e436a9b19ca130b8d91ce, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:40.950409 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1b257b-58b6-4bec-8786-b7c5ef6f2368 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:40.968916 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1073/2151] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:27:40 2022] PUT /identity/v3/projects/3f37e42e01664a5eb97e7b3f52b8c6f9/users/43402622442e436a9b19ca130b8d91ce/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:40.979748 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689be2d8-e9d1-4c74-9d06-c4ebbd0b5381 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:40.980038 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689be2d8-e9d1-4c74-9d06-c4ebbd0b5381 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:40.980320 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689be2d8-e9d1-4c74-9d06-c4ebbd0b5381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:41.009967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1079/2152] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:40 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:27:41.014202 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-689be2d8-e9d1-4c74-9d06-c4ebbd0b5381 None None] MFA Rules not processed for user `43402622442e436a9b19ca130b8d91ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:41.015447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-689be2d8-e9d1-4c74-9d06-c4ebbd0b5381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:41.023255 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764fed2c-6739-4177-b137-64302e0c8970 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:41.095100 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764fed2c-6739-4177-b137-64302e0c8970 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:41.115561 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1074/2153] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:40 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:27:41.120814 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689be2d8-e9d1-4c74-9d06-c4ebbd0b5381 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:41.136234 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:41.156461 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22053ff2-a7af-4a51-9637-78b16d28a819 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:41.158093 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22053ff2-a7af-4a51-9637-78b16d28a819 service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:41.158365 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22053ff2-a7af-4a51-9637-78b16d28a819 service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:41.158643 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22053ff2-a7af-4a51-9637-78b16d28a819 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:41.213881 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689be2d8-e9d1-4c74-9d06-c4ebbd0b5381 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:41.219685 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177d9e3-1420-4e42-b9f0-04caf35056d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:41.220304 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177d9e3-1420-4e42-b9f0-04caf35056d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:41.220662 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177d9e3-1420-4e42-b9f0-04caf35056d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:41.220875 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177d9e3-1420-4e42-b9f0-04caf35056d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:41.221527 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177d9e3-1420-4e42-b9f0-04caf35056d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:41.223636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177d9e3-1420-4e42-b9f0-04caf35056d3 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:41.237177 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22053ff2-a7af-4a51-9637-78b16d28a819 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:41.238849 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22053ff2-a7af-4a51-9637-78b16d28a819 service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:41.249980 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1080/2154] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:27:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:27:41.261518 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1075/2155] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:27:41 2022] POST /identity/v3/projects => generated 385 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:27:41.266315 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22053ff2-a7af-4a51-9637-78b16d28a819 service cinder] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:41.280733 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22053ff2-a7af-4a51-9637-78b16d28a819 service cinder] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:41.356151 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0fef42-f0d7-420b-aed8-0c580e04d1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:41.357009 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0fef42-f0d7-420b-aed8-0c580e04d1e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:41.357434 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0fef42-f0d7-420b-aed8-0c580e04d1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:41.357711 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0fef42-f0d7-420b-aed8-0c580e04d1e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:41.358594 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0fef42-f0d7-420b-aed8-0c580e04d1e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:41.360809 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0fef42-f0d7-420b-aed8-0c580e04d1e0 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:41.422860 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1081/2156] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:27:41 2022] POST /identity/v3/users => generated 299 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:27:41.424772 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177d9e3-1420-4e42-b9f0-04caf35056d3 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:41.439854 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177d9e3-1420-4e42-b9f0-04caf35056d3 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:41.447961 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc2c28-b7a7-4a83-923d-98bfacb6cb6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:41.448671 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc2c28-b7a7-4a83-923d-98bfacb6cb6a admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:41.449068 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc2c28-b7a7-4a83-923d-98bfacb6cb6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:41.449433 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc2c28-b7a7-4a83-923d-98bfacb6cb6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:41.450298 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cc2c28-b7a7-4a83-923d-98bfacb6cb6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:41.452154 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cc2c28-b7a7-4a83-923d-98bfacb6cb6a admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:41.484823 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1076/2157] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:41 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 14:27:41.486465 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0fef42-f0d7-420b-aed8-0c580e04d1e0 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:41.501777 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0fef42-f0d7-420b-aed8-0c580e04d1e0 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:41.509084 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf501f2d-e1ec-49fa-ae07-21e9672a6c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:41.510073 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf501f2d-e1ec-49fa-ae07-21e9672a6c17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:41.510454 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf501f2d-e1ec-49fa-ae07-21e9672a6c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:41.511123 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf501f2d-e1ec-49fa-ae07-21e9672a6c17 admin admin] PATH_INFO: `/v3/projects/44c7d33290854b6ab828c86c2c4c10b2/users/f8af518ff36c4c9ab9d965b1e6242358/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:41.545846 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf501f2d-e1ec-49fa-ae07-21e9672a6c17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44c7d33290854b6ab828c86c2c4c10b2, user_id=f8af518ff36c4c9ab9d965b1e6242358, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:41.556061 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf501f2d-e1ec-49fa-ae07-21e9672a6c17 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:41.582877 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1082/2158] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:27:41 2022] PUT /identity/v3/projects/44c7d33290854b6ab828c86c2c4c10b2/users/f8af518ff36c4c9ab9d965b1e6242358/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:41.587329 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc2c28-b7a7-4a83-923d-98bfacb6cb6a admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:41.603382 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc2c28-b7a7-4a83-923d-98bfacb6cb6a admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:41.627202 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:41.725124 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c28434b-33ce-4616-a0cd-00d64df4383c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:41.725124 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28434b-33ce-4616-a0cd-00d64df4383c admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:41.725718 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28434b-33ce-4616-a0cd-00d64df4383c admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:41.726099 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28434b-33ce-4616-a0cd-00d64df4383c admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:41.727038 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c28434b-33ce-4616-a0cd-00d64df4383c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:41.728765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c28434b-33ce-4616-a0cd-00d64df4383c admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:41.742098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1077/2159] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:41 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:27:41.747413 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf501f2d-e1ec-49fa-ae07-21e9672a6c17 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:41.771651 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf501f2d-e1ec-49fa-ae07-21e9672a6c17 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:41.781463 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a5ec7c-23fb-4c07-8a0a-d4d9961cebb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:41.782385 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a5ec7c-23fb-4c07-8a0a-d4d9961cebb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:41.795034 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a5ec7c-23fb-4c07-8a0a-d4d9961cebb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:41.795034 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a5ec7c-23fb-4c07-8a0a-d4d9961cebb7 admin admin] PATH_INFO: `/v3/projects/44c7d33290854b6ab828c86c2c4c10b2/users/f8af518ff36c4c9ab9d965b1e6242358/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:41.804643 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a5ec7c-23fb-4c07-8a0a-d4d9961cebb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44c7d33290854b6ab828c86c2c4c10b2, user_id=f8af518ff36c4c9ab9d965b1e6242358, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:41.807720 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a5ec7c-23fb-4c07-8a0a-d4d9961cebb7 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:41.859989 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1083/2160] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:27:41 2022] PUT /identity/v3/projects/44c7d33290854b6ab828c86c2c4c10b2/users/f8af518ff36c4c9ab9d965b1e6242358/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:27:41.872727 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcbb642-7abe-4795-af21-5a387a93d3e3 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:41.873007 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcbb642-7abe-4795-af21-5a387a93d3e3 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:41.873284 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcbb642-7abe-4795-af21-5a387a93d3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:41.901062 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-2fcbb642-7abe-4795-af21-5a387a93d3e3 None None] MFA Rules not processed for user `f8af518ff36c4c9ab9d965b1e6242358`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:41.902261 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-2fcbb642-7abe-4795-af21-5a387a93d3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:41.987493 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1078/2161] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:41 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:27:41.998727 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a5ec7c-23fb-4c07-8a0a-d4d9961cebb7 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:42.045244 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:42.058504 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a5ec7c-23fb-4c07-8a0a-d4d9961cebb7 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:42.062354 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a6952-2975-4aac-9b9a-311c389a119b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:42.063015 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a6952-2975-4aac-9b9a-311c389a119b service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:42.063300 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a6952-2975-4aac-9b9a-311c389a119b service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:42.063573 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a6952-2975-4aac-9b9a-311c389a119b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:42.073274 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1a6952-2975-4aac-9b9a-311c389a119b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:42.074715 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1a6952-2975-4aac-9b9a-311c389a119b service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:42.084911 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1084/2162] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:27:41 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:27:42.644463 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcbb642-7abe-4795-af21-5a387a93d3e3 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:42.654218 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcbb642-7abe-4795-af21-5a387a93d3e3 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:42.663193 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb0ca98-e57a-474f-b965-f29ba79a95a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:42.664384 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0ca98-e57a-474f-b965-f29ba79a95a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:42.664836 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0ca98-e57a-474f-b965-f29ba79a95a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:42.665114 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0ca98-e57a-474f-b965-f29ba79a95a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:42.729886 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb0ca98-e57a-474f-b965-f29ba79a95a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:42.731737 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb0ca98-e57a-474f-b965-f29ba79a95a0 service cinder] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:42.742718 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1079/2163] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:27:42 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:27:49.302561 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a6952-2975-4aac-9b9a-311c389a119b service cinder] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:49.367675 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a6952-2975-4aac-9b9a-311c389a119b service cinder] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:49.372132 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d956c1b-50b5-4427-8676-861fd5423f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:49.372744 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d956c1b-50b5-4427-8676-861fd5423f68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:49.373019 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d956c1b-50b5-4427-8676-861fd5423f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:49.373290 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d956c1b-50b5-4427-8676-861fd5423f68 admin admin] PATH_INFO: `/v3/users/81dbf0922d4d4b62aa6c63fd92396c72` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:49.374384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d956c1b-50b5-4427-8676-861fd5423f68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81dbf0922d4d4b62aa6c63fd92396c72)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:49.376310 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d956c1b-50b5-4427-8676-861fd5423f68 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:49.423086 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-0d956c1b-50b5-4427-8676-861fd5423f68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81dbf0922d4d4b62aa6c63fd92396c72'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:49.423820 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-0d956c1b-50b5-4427-8676-861fd5423f68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81dbf0922d4d4b62aa6c63fd92396c72'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:49.427182 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-0d956c1b-50b5-4427-8676-861fd5423f68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81dbf0922d4d4b62aa6c63fd92396c72'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:49.439699 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-0d956c1b-50b5-4427-8676-861fd5423f68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81dbf0922d4d4b62aa6c63fd92396c72'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:49.448424 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-0d956c1b-50b5-4427-8676-861fd5423f68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81dbf0922d4d4b62aa6c63fd92396c72'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:49.463748 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1085/2164] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:27:49 2022] DELETE /identity/v3/users/81dbf0922d4d4b62aa6c63fd92396c72 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:49.468177 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e976a2-6a2e-4931-af7e-2bb8f476f5a3 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:49.468687 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e976a2-6a2e-4931-af7e-2bb8f476f5a3 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:49.468798 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e976a2-6a2e-4931-af7e-2bb8f476f5a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:49.488337 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-a5e976a2-6a2e-4931-af7e-2bb8f476f5a3 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:49.491790 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-a5e976a2-6a2e-4931-af7e-2bb8f476f5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:49.534997 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:49.580733 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1080/2165] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:27:49.594647 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d956c1b-50b5-4427-8676-861fd5423f68 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:49.596143 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-0d956c1b-50b5-4427-8676-861fd5423f68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:49.680326 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d956c1b-50b5-4427-8676-861fd5423f68 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:49.684340 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce93d972-79d6-4ec6-979f-a6d9fe66fca6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:49.685186 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce93d972-79d6-4ec6-979f-a6d9fe66fca6 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:49.685469 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce93d972-79d6-4ec6-979f-a6d9fe66fca6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:49.685998 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce93d972-79d6-4ec6-979f-a6d9fe66fca6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:49.697612 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce93d972-79d6-4ec6-979f-a6d9fe66fca6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:49.698850 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce93d972-79d6-4ec6-979f-a6d9fe66fca6 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:49.711649 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1086/2166] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:27:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:27:50.009343 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e976a2-6a2e-4931-af7e-2bb8f476f5a3 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:50.010869 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-a5e976a2-6a2e-4931-af7e-2bb8f476f5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:50.029113 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e976a2-6a2e-4931-af7e-2bb8f476f5a3 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:50.043544 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6255c1c7-207b-4ebc-a9cf-a1fff1a6c58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:50.044257 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255c1c7-207b-4ebc-a9cf-a1fff1a6c58e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:50.044549 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255c1c7-207b-4ebc-a9cf-a1fff1a6c58e admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:50.044832 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255c1c7-207b-4ebc-a9cf-a1fff1a6c58e admin admin] PATH_INFO: `/v3/projects/e19d1bcc7eca4c39842e754b7590e571` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:50.046019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6255c1c7-207b-4ebc-a9cf-a1fff1a6c58e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e19d1bcc7eca4c39842e754b7590e571)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:50.048393 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6255c1c7-207b-4ebc-a9cf-a1fff1a6c58e admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:50.111185 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-6255c1c7-207b-4ebc-a9cf-a1fff1a6c58e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e19d1bcc7eca4c39842e754b7590e571', 'request_id': 'req-6255c1c7-207b-4ebc-a9cf-a1fff1a6c58e'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:50.118415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-6255c1c7-207b-4ebc-a9cf-a1fff1a6c58e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e19d1bcc7eca4c39842e754b7590e571', 'request_id': 'req-6255c1c7-207b-4ebc-a9cf-a1fff1a6c58e'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:50.124510 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-6255c1c7-207b-4ebc-a9cf-a1fff1a6c58e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e19d1bcc7eca4c39842e754b7590e571', 'request_id': 'req-6255c1c7-207b-4ebc-a9cf-a1fff1a6c58e'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:50.138240 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-6255c1c7-207b-4ebc-a9cf-a1fff1a6c58e admin admin] The token cache is being invalidate because project e19d1bcc7eca4c39842e754b7590e571 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:27:50.138530 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-6255c1c7-207b-4ebc-a9cf-a1fff1a6c58e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:50.139168 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-6255c1c7-207b-4ebc-a9cf-a1fff1a6c58e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:50.140814 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1081/2167] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:27:50 2022] DELETE /identity/v3/projects/e19d1bcc7eca4c39842e754b7590e571 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:50.144826 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce93d972-79d6-4ec6-979f-a6d9fe66fca6 service neutron] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:50.155035 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce93d972-79d6-4ec6-979f-a6d9fe66fca6 service neutron] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:50.159016 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:50.218414 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bbf22d-7481-4d28-ba25-ea44d1197912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:50.219076 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bbf22d-7481-4d28-ba25-ea44d1197912 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:50.219353 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bbf22d-7481-4d28-ba25-ea44d1197912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:50.219622 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bbf22d-7481-4d28-ba25-ea44d1197912 admin admin] PATH_INFO: `/v3/users/71b66eb47d2f4b2c909215d72a9c9f3c` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:50.220720 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bbf22d-7481-4d28-ba25-ea44d1197912 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71b66eb47d2f4b2c909215d72a9c9f3c)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:50.222617 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bbf22d-7481-4d28-ba25-ea44d1197912 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:50.268738 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-51bbf22d-7481-4d28-ba25-ea44d1197912 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71b66eb47d2f4b2c909215d72a9c9f3c'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:50.269318 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-51bbf22d-7481-4d28-ba25-ea44d1197912 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71b66eb47d2f4b2c909215d72a9c9f3c'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:50.269923 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-51bbf22d-7481-4d28-ba25-ea44d1197912 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71b66eb47d2f4b2c909215d72a9c9f3c'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:50.286451 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-51bbf22d-7481-4d28-ba25-ea44d1197912 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71b66eb47d2f4b2c909215d72a9c9f3c'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:50.313758 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-51bbf22d-7481-4d28-ba25-ea44d1197912 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71b66eb47d2f4b2c909215d72a9c9f3c'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:50.329951 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1087/2168] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:27:50 2022] DELETE /identity/v3/users/71b66eb47d2f4b2c909215d72a9c9f3c => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:27:50.625172 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6255c1c7-207b-4ebc-a9cf-a1fff1a6c58e admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:50.626670 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-6255c1c7-207b-4ebc-a9cf-a1fff1a6c58e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:50.640294 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6255c1c7-207b-4ebc-a9cf-a1fff1a6c58e admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:50.710842 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e6ba1c-7708-4473-9296-b09701705ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:50.710842 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6ba1c-7708-4473-9296-b09701705ed4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:50.711318 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6ba1c-7708-4473-9296-b09701705ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:50.711660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6ba1c-7708-4473-9296-b09701705ed4 admin admin] PATH_INFO: `/v3/projects/f9a13890743242edae5996e02b625dde` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:50.713666 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e6ba1c-7708-4473-9296-b09701705ed4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9a13890743242edae5996e02b625dde)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:50.715721 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e6ba1c-7708-4473-9296-b09701705ed4 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:50.769668 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-63e6ba1c-7708-4473-9296-b09701705ed4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9a13890743242edae5996e02b625dde', 'request_id': 'req-63e6ba1c-7708-4473-9296-b09701705ed4'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:50.777277 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-63e6ba1c-7708-4473-9296-b09701705ed4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9a13890743242edae5996e02b625dde', 'request_id': 'req-63e6ba1c-7708-4473-9296-b09701705ed4'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:50.782420 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:50.784278 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-63e6ba1c-7708-4473-9296-b09701705ed4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9a13890743242edae5996e02b625dde', 'request_id': 'req-63e6ba1c-7708-4473-9296-b09701705ed4'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:50.807974 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-63e6ba1c-7708-4473-9296-b09701705ed4 admin admin] The token cache is being invalidate because project f9a13890743242edae5996e02b625dde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:27:50.808296 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-63e6ba1c-7708-4473-9296-b09701705ed4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:50.808827 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-63e6ba1c-7708-4473-9296-b09701705ed4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:50.810594 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1082/2169] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:27:50 2022] DELETE /identity/v3/projects/f9a13890743242edae5996e02b625dde => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:50.816888 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17fa43-9679-4554-bcf1-310b84ec5599 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:50.817345 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17fa43-9679-4554-bcf1-310b84ec5599 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:50.817733 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17fa43-9679-4554-bcf1-310b84ec5599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:50.842249 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-de17fa43-9679-4554-bcf1-310b84ec5599 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:50.843490 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-de17fa43-9679-4554-bcf1-310b84ec5599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:50.878147 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e6ba1c-7708-4473-9296-b09701705ed4 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:50.879823 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-63e6ba1c-7708-4473-9296-b09701705ed4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:50.934007 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1088/2170] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:27:50.942172 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac43fe-5d52-44c2-87f6-2e1cd77c6a5b None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:50.942471 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac43fe-5d52-44c2-87f6-2e1cd77c6a5b None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:50.942723 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac43fe-5d52-44c2-87f6-2e1cd77c6a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:50.966441 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-ffac43fe-5d52-44c2-87f6-2e1cd77c6a5b None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:50.967696 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-ffac43fe-5d52-44c2-87f6-2e1cd77c6a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:50.983158 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1089/2171] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:27:50.984377 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffac43fe-5d52-44c2-87f6-2e1cd77c6a5b None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:50.989916 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e6ba1c-7708-4473-9296-b09701705ed4 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:50.996774 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d1fc38-ce82-4a73-b619-bfadede36ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:50.996989 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffac43fe-5d52-44c2-87f6-2e1cd77c6a5b None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:50.998012 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1fc38-ce82-4a73-b619-bfadede36ab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:50.998473 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1fc38-ce82-4a73-b619-bfadede36ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:50.998952 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1fc38-ce82-4a73-b619-bfadede36ab8 admin admin] PATH_INFO: `/v3/users/06d4574818ee42abbb430981734518a5` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:51.000867 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d1fc38-ce82-4a73-b619-bfadede36ab8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06d4574818ee42abbb430981734518a5)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:51.003674 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d1fc38-ce82-4a73-b619-bfadede36ab8 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:51.012564 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed969b-0997-489a-bcff-1a1b04ffd231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:51.013282 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed969b-0997-489a-bcff-1a1b04ffd231 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:51.013590 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed969b-0997-489a-bcff-1a1b04ffd231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:51.013855 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed969b-0997-489a-bcff-1a1b04ffd231 admin admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:51.014514 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed969b-0997-489a-bcff-1a1b04ffd231 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:27:51.014922 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed969b-0997-489a-bcff-1a1b04ffd231 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:51.016883 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed969b-0997-489a-bcff-1a1b04ffd231 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:51.035715 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1090/2172] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:27:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:27:51.042276 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed969b-0997-489a-bcff-1a1b04ffd231 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:51.059191 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed969b-0997-489a-bcff-1a1b04ffd231 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:51.066473 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c90a55-1b4b-464c-b3f5-f76155429523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:51.067419 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c90a55-1b4b-464c-b3f5-f76155429523 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:51.067764 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c90a55-1b4b-464c-b3f5-f76155429523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:51.068101 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c90a55-1b4b-464c-b3f5-f76155429523 admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:51.069182 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c90a55-1b4b-464c-b3f5-f76155429523 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:51.071921 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c90a55-1b4b-464c-b3f5-f76155429523 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:51.077355 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:51.088293 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-63d1fc38-ce82-4a73-b619-bfadede36ab8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06d4574818ee42abbb430981734518a5'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:51.089046 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-63d1fc38-ce82-4a73-b619-bfadede36ab8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06d4574818ee42abbb430981734518a5'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:51.099400 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-63d1fc38-ce82-4a73-b619-bfadede36ab8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06d4574818ee42abbb430981734518a5'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:51.100609 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-63d1fc38-ce82-4a73-b619-bfadede36ab8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06d4574818ee42abbb430981734518a5'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:51.117753 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-63d1fc38-ce82-4a73-b619-bfadede36ab8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06d4574818ee42abbb430981734518a5'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:51.118399 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1091/2173] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:27:51 2022] POST /identity/v3/projects => generated 375 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:27:51.122901 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c90a55-1b4b-464c-b3f5-f76155429523 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:51.125080 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-98c90a55-1b4b-464c-b3f5-f76155429523 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:51.134078 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1083/2174] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:27:50 2022] DELETE /identity/v3/users/06d4574818ee42abbb430981734518a5 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:51.139051 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a9d2a-20b9-4bfc-910d-a44b80263076 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:51.139348 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a9d2a-20b9-4bfc-910d-a44b80263076 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:51.139609 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a9d2a-20b9-4bfc-910d-a44b80263076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:51.161002 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-e67a9d2a-20b9-4bfc-910d-a44b80263076 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:51.162196 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-e67a9d2a-20b9-4bfc-910d-a44b80263076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:51.197912 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c90a55-1b4b-464c-b3f5-f76155429523 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:51.244316 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1084/2175] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:27:51.257572 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a9d2a-20b9-4bfc-910d-a44b80263076 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:51.259301 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-e67a9d2a-20b9-4bfc-910d-a44b80263076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:51.265214 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c5f968-3d8a-41b1-86fb-729ef2ca81b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:51.267383 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5f968-3d8a-41b1-86fb-729ef2ca81b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:51.270528 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5f968-3d8a-41b1-86fb-729ef2ca81b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:51.270842 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5f968-3d8a-41b1-86fb-729ef2ca81b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:51.271515 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c5f968-3d8a-41b1-86fb-729ef2ca81b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:51.273563 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c5f968-3d8a-41b1-86fb-729ef2ca81b3 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:51.274128 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:51.305180 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1092/2176] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:27:51 2022] POST /identity/v3/users => generated 288 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:27:51.306664 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c5f968-3d8a-41b1-86fb-729ef2ca81b3 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:51.320972 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c5f968-3d8a-41b1-86fb-729ef2ca81b3 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:51.325995 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35f451d-3d80-44b6-9ffd-584f7888c9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:51.330924 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f451d-3d80-44b6-9ffd-584f7888c9cc admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:51.330924 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f451d-3d80-44b6-9ffd-584f7888c9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:51.330924 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f451d-3d80-44b6-9ffd-584f7888c9cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:51.330924 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35f451d-3d80-44b6-9ffd-584f7888c9cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:51.330924 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35f451d-3d80-44b6-9ffd-584f7888c9cc admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:51.342160 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1093/2177] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:27:51.346421 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35f451d-3d80-44b6-9ffd-584f7888c9cc admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:51.356332 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35f451d-3d80-44b6-9ffd-584f7888c9cc admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:51.357124 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a9d2a-20b9-4bfc-910d-a44b80263076 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:51.361096 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54808571-ee43-408d-b858-829fec1949d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:51.361814 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54808571-ee43-408d-b858-829fec1949d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:51.362117 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54808571-ee43-408d-b858-829fec1949d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:51.362232 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2f2a1-42b5-40a8-afb1-58b1ab1e48f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:51.362506 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54808571-ee43-408d-b858-829fec1949d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:51.363008 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2f2a1-42b5-40a8-afb1-58b1ab1e48f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:51.363362 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2f2a1-42b5-40a8-afb1-58b1ab1e48f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:51.363702 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2f2a1-42b5-40a8-afb1-58b1ab1e48f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:51.364425 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd2f2a1-42b5-40a8-afb1-58b1ab1e48f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:51.366976 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd2f2a1-42b5-40a8-afb1-58b1ab1e48f4 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:51.374365 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54808571-ee43-408d-b858-829fec1949d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:51.375839 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54808571-ee43-408d-b858-829fec1949d7 service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:51.384670 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1094/2178] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 14:27:51.385486 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2f2a1-42b5-40a8-afb1-58b1ab1e48f4 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:51.387164 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1085/2179] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:27:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 14:27:51.397813 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2f2a1-42b5-40a8-afb1-58b1ab1e48f4 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:51.403624 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1181431f-9337-4e17-be0c-a74a03ee7f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:51.404290 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181431f-9337-4e17-be0c-a74a03ee7f4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:51.404570 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181431f-9337-4e17-be0c-a74a03ee7f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:51.404840 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181431f-9337-4e17-be0c-a74a03ee7f4b admin admin] PATH_INFO: `/v3/projects/bcb7ad6606444deaacc1012ba6e49268/users/068490a4a67147c8967f1e2ee9b4cf2b/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:51.422891 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1181431f-9337-4e17-be0c-a74a03ee7f4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcb7ad6606444deaacc1012ba6e49268, user_id=068490a4a67147c8967f1e2ee9b4cf2b, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:51.426795 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1181431f-9337-4e17-be0c-a74a03ee7f4b admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:51.450635 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1095/2180] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:27:51 2022] PUT /identity/v3/projects/bcb7ad6606444deaacc1012ba6e49268/users/068490a4a67147c8967f1e2ee9b4cf2b/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:51.469561 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bdc850-9c33-4879-84ef-ec1244c8f46f None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:51.470102 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bdc850-9c33-4879-84ef-ec1244c8f46f None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:51.470427 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bdc850-9c33-4879-84ef-ec1244c8f46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:51.520584 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-b9bdc850-9c33-4879-84ef-ec1244c8f46f None None] MFA Rules not processed for user `068490a4a67147c8967f1e2ee9b4cf2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:51.521816 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-b9bdc850-9c33-4879-84ef-ec1244c8f46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:51.617561 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1086/2181] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:27:51.628683 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1181431f-9337-4e17-be0c-a74a03ee7f4b admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:51.630836 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-1181431f-9337-4e17-be0c-a74a03ee7f4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:51.701122 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:51.725946 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1181431f-9337-4e17-be0c-a74a03ee7f4b admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:51.732432 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cdf9cb-fee7-4d83-ae5b-b33439bf2bd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:51.733067 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cdf9cb-fee7-4d83-ae5b-b33439bf2bd0 service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:51.733345 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cdf9cb-fee7-4d83-ae5b-b33439bf2bd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:51.733632 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cdf9cb-fee7-4d83-ae5b-b33439bf2bd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:51.747137 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cdf9cb-fee7-4d83-ae5b-b33439bf2bd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:51.749860 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cdf9cb-fee7-4d83-ae5b-b33439bf2bd0 service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:51.763991 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1096/2182] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:27:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 14:27:51.764190 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bdc850-9c33-4879-84ef-ec1244c8f46f None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:51.764190 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-b9bdc850-9c33-4879-84ef-ec1244c8f46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:51.856941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bdc850-9c33-4879-84ef-ec1244c8f46f None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:51.863276 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7ba55-d5fa-44b7-a1f5-e5431fcd14ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:51.864238 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7ba55-d5fa-44b7-a1f5-e5431fcd14ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:51.864522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7ba55-d5fa-44b7-a1f5-e5431fcd14ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:51.864798 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7ba55-d5fa-44b7-a1f5-e5431fcd14ad admin admin] PATH_INFO: `/v3/projects/f46ad7a1a22049b0a6bef212498844e9` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:51.866164 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b7ba55-d5fa-44b7-a1f5-e5431fcd14ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=f46ad7a1a22049b0a6bef212498844e9)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:51.868848 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b7ba55-d5fa-44b7-a1f5-e5431fcd14ad admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:51.921542 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:51.926458 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-15b7ba55-d5fa-44b7-a1f5-e5431fcd14ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f46ad7a1a22049b0a6bef212498844e9', 'request_id': 'req-15b7ba55-d5fa-44b7-a1f5-e5431fcd14ad'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:51.935150 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-15b7ba55-d5fa-44b7-a1f5-e5431fcd14ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f46ad7a1a22049b0a6bef212498844e9', 'request_id': 'req-15b7ba55-d5fa-44b7-a1f5-e5431fcd14ad'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:51.942062 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-15b7ba55-d5fa-44b7-a1f5-e5431fcd14ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f46ad7a1a22049b0a6bef212498844e9', 'request_id': 'req-15b7ba55-d5fa-44b7-a1f5-e5431fcd14ad'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:51.955791 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-15b7ba55-d5fa-44b7-a1f5-e5431fcd14ad admin admin] The token cache is being invalidate because project f46ad7a1a22049b0a6bef212498844e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:27:51.956429 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-15b7ba55-d5fa-44b7-a1f5-e5431fcd14ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:51.957334 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-15b7ba55-d5fa-44b7-a1f5-e5431fcd14ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:51.960441 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1087/2183] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:27:51 2022] DELETE /identity/v3/projects/f46ad7a1a22049b0a6bef212498844e9 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:51.965286 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cdf9cb-fee7-4d83-ae5b-b33439bf2bd0 service cinder] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:51.977015 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cdf9cb-fee7-4d83-ae5b-b33439bf2bd0 service cinder] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:52.043649 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ce5ab-5393-4673-a260-8ea15a5396a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:52.044287 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ce5ab-5393-4673-a260-8ea15a5396a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:52.044569 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ce5ab-5393-4673-a260-8ea15a5396a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:52.044847 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ce5ab-5393-4673-a260-8ea15a5396a0 admin admin] PATH_INFO: `/v3/users/01f08201de2944d0a79ebc8de0d13eb9` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:52.045975 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ce5ab-5393-4673-a260-8ea15a5396a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01f08201de2944d0a79ebc8de0d13eb9)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:52.048799 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ce5ab-5393-4673-a260-8ea15a5396a0 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:52.152362 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d41ce5ab-5393-4673-a260-8ea15a5396a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01f08201de2944d0a79ebc8de0d13eb9'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:52.153485 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d41ce5ab-5393-4673-a260-8ea15a5396a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01f08201de2944d0a79ebc8de0d13eb9'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:52.154128 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d41ce5ab-5393-4673-a260-8ea15a5396a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01f08201de2944d0a79ebc8de0d13eb9'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:52.182781 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d41ce5ab-5393-4673-a260-8ea15a5396a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01f08201de2944d0a79ebc8de0d13eb9'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:52.190424 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d41ce5ab-5393-4673-a260-8ea15a5396a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01f08201de2944d0a79ebc8de0d13eb9'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:52.204439 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1097/2184] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:27:51 2022] DELETE /identity/v3/users/01f08201de2944d0a79ebc8de0d13eb9 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:52.332657 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7ba55-d5fa-44b7-a1f5-e5431fcd14ad admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:52.334276 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-15b7ba55-d5fa-44b7-a1f5-e5431fcd14ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:52.400607 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7ba55-d5fa-44b7-a1f5-e5431fcd14ad admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:52.455846 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c3588-4ed5-423f-ba31-6becc069aadd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:52.456772 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c3588-4ed5-423f-ba31-6becc069aadd service cinder] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:52.457210 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c3588-4ed5-423f-ba31-6becc069aadd service cinder] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:52.457485 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c3588-4ed5-423f-ba31-6becc069aadd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:52.459273 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-d29c3588-4ed5-423f-ba31-6becc069aadd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:52.527660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c3588-4ed5-423f-ba31-6becc069aadd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:52.529107 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c3588-4ed5-423f-ba31-6becc069aadd service cinder] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:52.539738 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1088/2185] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:27:52 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:27:52.590321 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ce5ab-5393-4673-a260-8ea15a5396a0 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:52.591840 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-d41ce5ab-5393-4673-a260-8ea15a5396a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:52.670061 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:52.678847 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ce5ab-5393-4673-a260-8ea15a5396a0 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:52.688120 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252940a-e573-41d9-895d-aced1b4d6bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:52.688868 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252940a-e573-41d9-895d-aced1b4d6bdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:52.689466 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252940a-e573-41d9-895d-aced1b4d6bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:52.689765 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252940a-e573-41d9-895d-aced1b4d6bdc admin admin] PATH_INFO: `/v3/projects/ca8bd83bab374e708b66af809461c326` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:52.691576 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4252940a-e573-41d9-895d-aced1b4d6bdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca8bd83bab374e708b66af809461c326)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:52.693820 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4252940a-e573-41d9-895d-aced1b4d6bdc admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:52.746399 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-4252940a-e573-41d9-895d-aced1b4d6bdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca8bd83bab374e708b66af809461c326', 'request_id': 'req-4252940a-e573-41d9-895d-aced1b4d6bdc'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:52.753347 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-4252940a-e573-41d9-895d-aced1b4d6bdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca8bd83bab374e708b66af809461c326', 'request_id': 'req-4252940a-e573-41d9-895d-aced1b4d6bdc'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:52.760455 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-4252940a-e573-41d9-895d-aced1b4d6bdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca8bd83bab374e708b66af809461c326', 'request_id': 'req-4252940a-e573-41d9-895d-aced1b4d6bdc'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:52.774929 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-4252940a-e573-41d9-895d-aced1b4d6bdc admin admin] The token cache is being invalidate because project ca8bd83bab374e708b66af809461c326 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:27:52.775252 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-4252940a-e573-41d9-895d-aced1b4d6bdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:52.775873 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-4252940a-e573-41d9-895d-aced1b4d6bdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:52.777649 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1098/2186] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:27:52 2022] DELETE /identity/v3/projects/ca8bd83bab374e708b66af809461c326 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:27:52.783877 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257fdff0-a04b-4c42-b927-0db9475b4042 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:52.784337 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257fdff0-a04b-4c42-b927-0db9475b4042 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:52.784427 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257fdff0-a04b-4c42-b927-0db9475b4042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:52.805777 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-257fdff0-a04b-4c42-b927-0db9475b4042 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:52.807006 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-257fdff0-a04b-4c42-b927-0db9475b4042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:52.851800 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:52.879420 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1089/2187] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:27:52.887936 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237acb19-1541-411d-9742-54cc1192682c None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:52.888222 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237acb19-1541-411d-9742-54cc1192682c None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:52.888514 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237acb19-1541-411d-9742-54cc1192682c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:52.908285 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-237acb19-1541-411d-9742-54cc1192682c None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:52.909753 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-237acb19-1541-411d-9742-54cc1192682c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:52.918383 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1099/2188] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:27:52.924155 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257fdff0-a04b-4c42-b927-0db9475b4042 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:52.936051 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257fdff0-a04b-4c42-b927-0db9475b4042 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:52.941817 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207949c9-1f19-4cba-8728-2081e0691e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:52.942443 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207949c9-1f19-4cba-8728-2081e0691e59 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:52.942711 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207949c9-1f19-4cba-8728-2081e0691e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:52.943015 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207949c9-1f19-4cba-8728-2081e0691e59 admin admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:52.943818 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207949c9-1f19-4cba-8728-2081e0691e59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:27:52.944069 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207949c9-1f19-4cba-8728-2081e0691e59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:52.946228 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207949c9-1f19-4cba-8728-2081e0691e59 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:52.965236 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1090/2189] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:27:52.966898 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237acb19-1541-411d-9742-54cc1192682c None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:52.977258 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237acb19-1541-411d-9742-54cc1192682c None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:52.982807 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769e9702-a4fd-492c-bdc8-04ed3fe4e0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:52.983540 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769e9702-a4fd-492c-bdc8-04ed3fe4e0fd admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:52.983851 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769e9702-a4fd-492c-bdc8-04ed3fe4e0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:52.984150 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769e9702-a4fd-492c-bdc8-04ed3fe4e0fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:52.984858 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769e9702-a4fd-492c-bdc8-04ed3fe4e0fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:52.987198 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769e9702-a4fd-492c-bdc8-04ed3fe4e0fd admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:53.019978 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1100/2190] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:27:52 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:27:53.026108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207949c9-1f19-4cba-8728-2081e0691e59 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:53.039905 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207949c9-1f19-4cba-8728-2081e0691e59 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:53.109086 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c5c50-52b8-4a79-86f1-621d28c2b446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:53.109712 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c5c50-52b8-4a79-86f1-621d28c2b446 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:53.110072 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c5c50-52b8-4a79-86f1-621d28c2b446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:53.110406 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c5c50-52b8-4a79-86f1-621d28c2b446 admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:53.111069 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c5c50-52b8-4a79-86f1-621d28c2b446 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:53.112979 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c5c50-52b8-4a79-86f1-621d28c2b446 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:53.138744 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1091/2191] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:27:53 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:27:53.140310 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769e9702-a4fd-492c-bdc8-04ed3fe4e0fd admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:53.151780 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769e9702-a4fd-492c-bdc8-04ed3fe4e0fd admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:53.159715 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8311ad6-b623-4ea7-a31e-2332362a1e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:53.160328 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8311ad6-b623-4ea7-a31e-2332362a1e1d admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:53.160610 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8311ad6-b623-4ea7-a31e-2332362a1e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:53.160879 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8311ad6-b623-4ea7-a31e-2332362a1e1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:53.164022 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8311ad6-b623-4ea7-a31e-2332362a1e1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:53.164022 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8311ad6-b623-4ea7-a31e-2332362a1e1d admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:53.178564 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1101/2192] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:27:53.185185 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c5c50-52b8-4a79-86f1-621d28c2b446 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:53.195321 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c5c50-52b8-4a79-86f1-621d28c2b446 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:53.200097 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f91e50-2dca-4f12-8ecb-d2a791b8c6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:53.200726 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f91e50-2dca-4f12-8ecb-d2a791b8c6b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:53.201060 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f91e50-2dca-4f12-8ecb-d2a791b8c6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:53.201332 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f91e50-2dca-4f12-8ecb-d2a791b8c6b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:53.202040 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f91e50-2dca-4f12-8ecb-d2a791b8c6b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:53.203752 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f91e50-2dca-4f12-8ecb-d2a791b8c6b5 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:53.215069 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1092/2193] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:27:53.225989 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8311ad6-b623-4ea7-a31e-2332362a1e1d admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:53.239062 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8311ad6-b623-4ea7-a31e-2332362a1e1d admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:53.246043 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff0dec-fe57-4e42-8d86-20bee0702514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:53.246714 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff0dec-fe57-4e42-8d86-20bee0702514 admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:53.247071 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff0dec-fe57-4e42-8d86-20bee0702514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:53.247355 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff0dec-fe57-4e42-8d86-20bee0702514 admin admin] PATH_INFO: `/v3/projects/2b916ad697234d9c9562efe5d9e111c7/users/777ba964e39048f2a7f6e00349a9e465/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:53.271245 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ff0dec-fe57-4e42-8d86-20bee0702514 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b916ad697234d9c9562efe5d9e111c7, user_id=777ba964e39048f2a7f6e00349a9e465, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:53.272516 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ff0dec-fe57-4e42-8d86-20bee0702514 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:53.288845 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1102/2194] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:27:53 2022] PUT /identity/v3/projects/2b916ad697234d9c9562efe5d9e111c7/users/777ba964e39048f2a7f6e00349a9e465/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:53.299355 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3821693-82f7-418c-b23c-cb5ee1a9f434 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:53.299882 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3821693-82f7-418c-b23c-cb5ee1a9f434 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:53.300564 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3821693-82f7-418c-b23c-cb5ee1a9f434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:53.345952 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-f3821693-82f7-418c-b23c-cb5ee1a9f434 None None] MFA Rules not processed for user `777ba964e39048f2a7f6e00349a9e465`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:53.347291 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-f3821693-82f7-418c-b23c-cb5ee1a9f434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:53.436354 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1093/2195] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:27:53.448800 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff0dec-fe57-4e42-8d86-20bee0702514 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:53.450610 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-e7ff0dec-fe57-4e42-8d86-20bee0702514 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:53.512089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff0dec-fe57-4e42-8d86-20bee0702514 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:53.515818 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a1c2f3-3472-4609-9cff-d239dac8fcf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:53.516428 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1c2f3-3472-4609-9cff-d239dac8fcf4 service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:53.516715 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1c2f3-3472-4609-9cff-d239dac8fcf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:53.516988 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1c2f3-3472-4609-9cff-d239dac8fcf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:53.525811 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a1c2f3-3472-4609-9cff-d239dac8fcf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:53.527287 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a1c2f3-3472-4609-9cff-d239dac8fcf4 service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:53.537217 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1103/2196] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:27:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:27:54.869470 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3821693-82f7-418c-b23c-cb5ee1a9f434 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:54.870993 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-f3821693-82f7-418c-b23c-cb5ee1a9f434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:54.912715 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:54.938779 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3821693-82f7-418c-b23c-cb5ee1a9f434 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:54.943472 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99fa748-ddd2-451c-8bd1-9f16af49c247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:54.944157 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99fa748-ddd2-451c-8bd1-9f16af49c247 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:54.944790 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99fa748-ddd2-451c-8bd1-9f16af49c247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:54.945182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99fa748-ddd2-451c-8bd1-9f16af49c247 admin admin] PATH_INFO: `/v3/users/286d3cc6b18a49419cd9174d15a923e0` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:54.946407 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99fa748-ddd2-451c-8bd1-9f16af49c247 admin admin] RBAC: Authorizing `identity:delete_user(user_id=286d3cc6b18a49419cd9174d15a923e0)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:54.948330 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99fa748-ddd2-451c-8bd1-9f16af49c247 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:55.002213 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-d99fa748-ddd2-451c-8bd1-9f16af49c247 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '286d3cc6b18a49419cd9174d15a923e0'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:55.003002 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-d99fa748-ddd2-451c-8bd1-9f16af49c247 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '286d3cc6b18a49419cd9174d15a923e0'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:55.010454 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-d99fa748-ddd2-451c-8bd1-9f16af49c247 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '286d3cc6b18a49419cd9174d15a923e0'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:55.011266 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-d99fa748-ddd2-451c-8bd1-9f16af49c247 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '286d3cc6b18a49419cd9174d15a923e0'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:55.024198 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-d99fa748-ddd2-451c-8bd1-9f16af49c247 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '286d3cc6b18a49419cd9174d15a923e0'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:55.036915 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1094/2197] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:27:54 2022] DELETE /identity/v3/users/286d3cc6b18a49419cd9174d15a923e0 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:55.041875 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace39649-862d-468b-99dc-5a0811c667c5 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:55.042324 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace39649-862d-468b-99dc-5a0811c667c5 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:55.042324 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace39649-862d-468b-99dc-5a0811c667c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:55.061452 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-ace39649-862d-468b-99dc-5a0811c667c5 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:55.062777 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-ace39649-862d-468b-99dc-5a0811c667c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:55.065234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99fa748-ddd2-451c-8bd1-9f16af49c247 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:55.068014 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-d99fa748-ddd2-451c-8bd1-9f16af49c247 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:55.091422 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:55.150803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1104/2198] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:27:55.154509 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99fa748-ddd2-451c-8bd1-9f16af49c247 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:55.162124 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2001479-df2d-4835-ba04-0ae449568264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:55.164075 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2001479-df2d-4835-ba04-0ae449568264 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:55.164511 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2001479-df2d-4835-ba04-0ae449568264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:55.164934 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2001479-df2d-4835-ba04-0ae449568264 admin admin] PATH_INFO: `/v3/users/068490a4a67147c8967f1e2ee9b4cf2b` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:55.165262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace39649-862d-468b-99dc-5a0811c667c5 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:55.166717 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-ace39649-862d-468b-99dc-5a0811c667c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:55.167369 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2001479-df2d-4835-ba04-0ae449568264 admin admin] RBAC: Authorizing `identity:delete_user(user_id=068490a4a67147c8967f1e2ee9b4cf2b)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:55.169605 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2001479-df2d-4835-ba04-0ae449568264 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:55.242340 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-d2001479-df2d-4835-ba04-0ae449568264 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '068490a4a67147c8967f1e2ee9b4cf2b'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:55.243343 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-d2001479-df2d-4835-ba04-0ae449568264 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '068490a4a67147c8967f1e2ee9b4cf2b'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:55.247210 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace39649-862d-468b-99dc-5a0811c667c5 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:55.256103 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b3c9e-ff53-422e-b360-e776d60bfc63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:55.257063 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b3c9e-ff53-422e-b360-e776d60bfc63 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:55.257349 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b3c9e-ff53-422e-b360-e776d60bfc63 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:55.257627 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b3c9e-ff53-422e-b360-e776d60bfc63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:55.259510 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-450b3c9e-ff53-422e-b360-e776d60bfc63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:55.271665 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-d2001479-df2d-4835-ba04-0ae449568264 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '068490a4a67147c8967f1e2ee9b4cf2b'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:55.272941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-d2001479-df2d-4835-ba04-0ae449568264 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '068490a4a67147c8967f1e2ee9b4cf2b'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:55.275802 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:55.289439 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b3c9e-ff53-422e-b360-e776d60bfc63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:55.290447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-d2001479-df2d-4835-ba04-0ae449568264 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '068490a4a67147c8967f1e2ee9b4cf2b'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:55.291037 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b3c9e-ff53-422e-b360-e776d60bfc63 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:55.305595 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1105/2199] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:27:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:27:55.318477 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1095/2200] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:27:55 2022] DELETE /identity/v3/users/068490a4a67147c8967f1e2ee9b4cf2b => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:27:55.322984 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08152d00-5970-4bb8-999f-0954ccb1d71c None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:55.323282 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08152d00-5970-4bb8-999f-0954ccb1d71c None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:55.323557 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08152d00-5970-4bb8-999f-0954ccb1d71c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:55.366027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-08152d00-5970-4bb8-999f-0954ccb1d71c None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:55.367406 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-08152d00-5970-4bb8-999f-0954ccb1d71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:55.441377 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1106/2201] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:27:55.488037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2001479-df2d-4835-ba04-0ae449568264 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:55.488956 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-d2001479-df2d-4835-ba04-0ae449568264 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:55.606398 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2001479-df2d-4835-ba04-0ae449568264 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:55.613594 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ce83a-fadd-4cc8-bf2f-75f6437aa747 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:55.613594 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ce83a-fadd-4cc8-bf2f-75f6437aa747 service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:55.613787 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ce83a-fadd-4cc8-bf2f-75f6437aa747 service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:55.614204 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ce83a-fadd-4cc8-bf2f-75f6437aa747 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:55.636947 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ce83a-fadd-4cc8-bf2f-75f6437aa747 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:55.640523 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ce83a-fadd-4cc8-bf2f-75f6437aa747 service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:55.667691 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1096/2202] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:27:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:27:55.951893 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08152d00-5970-4bb8-999f-0954ccb1d71c None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:55.953423 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-08152d00-5970-4bb8-999f-0954ccb1d71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:55.970830 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08152d00-5970-4bb8-999f-0954ccb1d71c None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:55.976332 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a15cb00-747a-4571-948e-99bd92098fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:55.977049 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a15cb00-747a-4571-948e-99bd92098fa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:55.977391 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a15cb00-747a-4571-948e-99bd92098fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:55.977727 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a15cb00-747a-4571-948e-99bd92098fa1 admin admin] PATH_INFO: `/v3/projects/ad9c3603c7224fdc997699d08317cf21` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:55.979107 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a15cb00-747a-4571-948e-99bd92098fa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad9c3603c7224fdc997699d08317cf21)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:55.981666 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a15cb00-747a-4571-948e-99bd92098fa1 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:56.028412 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ce83a-fadd-4cc8-bf2f-75f6437aa747 service neutron] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:56.031086 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-c56ce83a-fadd-4cc8-bf2f-75f6437aa747 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:56.050872 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-3a15cb00-747a-4571-948e-99bd92098fa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad9c3603c7224fdc997699d08317cf21', 'request_id': 'req-3a15cb00-747a-4571-948e-99bd92098fa1'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:56.062577 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-3a15cb00-747a-4571-948e-99bd92098fa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad9c3603c7224fdc997699d08317cf21', 'request_id': 'req-3a15cb00-747a-4571-948e-99bd92098fa1'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:56.070243 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ce83a-fadd-4cc8-bf2f-75f6437aa747 service neutron] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:56.070750 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-3a15cb00-747a-4571-948e-99bd92098fa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad9c3603c7224fdc997699d08317cf21', 'request_id': 'req-3a15cb00-747a-4571-948e-99bd92098fa1'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:56.074601 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:56.096709 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-3a15cb00-747a-4571-948e-99bd92098fa1 admin admin] The token cache is being invalidate because project ad9c3603c7224fdc997699d08317cf21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:27:56.097142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-3a15cb00-747a-4571-948e-99bd92098fa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:56.098388 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-3a15cb00-747a-4571-948e-99bd92098fa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:56.099979 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1107/2203] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:27:55 2022] DELETE /identity/v3/projects/ad9c3603c7224fdc997699d08317cf21 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:56.104537 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a15cb00-747a-4571-948e-99bd92098fa1 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:56.105929 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-3a15cb00-747a-4571-948e-99bd92098fa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:56.154851 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a3ff0-6cc4-4f18-8b34-cfa3087df039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:56.155602 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a3ff0-6cc4-4f18-8b34-cfa3087df039 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:56.156691 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a3ff0-6cc4-4f18-8b34-cfa3087df039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:56.156691 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a3ff0-6cc4-4f18-8b34-cfa3087df039 admin admin] PATH_INFO: `/v3/projects/bcb7ad6606444deaacc1012ba6e49268` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:56.158566 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a3ff0-6cc4-4f18-8b34-cfa3087df039 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcb7ad6606444deaacc1012ba6e49268)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:56.160554 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a3ff0-6cc4-4f18-8b34-cfa3087df039 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:56.209912 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a15cb00-747a-4571-948e-99bd92098fa1 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:56.212885 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-744a3ff0-6cc4-4f18-8b34-cfa3087df039 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcb7ad6606444deaacc1012ba6e49268', 'request_id': 'req-744a3ff0-6cc4-4f18-8b34-cfa3087df039'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:56.223145 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-744a3ff0-6cc4-4f18-8b34-cfa3087df039 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcb7ad6606444deaacc1012ba6e49268', 'request_id': 'req-744a3ff0-6cc4-4f18-8b34-cfa3087df039'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:56.234372 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-744a3ff0-6cc4-4f18-8b34-cfa3087df039 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcb7ad6606444deaacc1012ba6e49268', 'request_id': 'req-744a3ff0-6cc4-4f18-8b34-cfa3087df039'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:56.252253 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-744a3ff0-6cc4-4f18-8b34-cfa3087df039 admin admin] The token cache is being invalidate because project bcb7ad6606444deaacc1012ba6e49268 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:27:56.252565 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-744a3ff0-6cc4-4f18-8b34-cfa3087df039 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:56.254204 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-744a3ff0-6cc4-4f18-8b34-cfa3087df039 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:56.256537 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1097/2204] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:27:56 2022] DELETE /identity/v3/projects/bcb7ad6606444deaacc1012ba6e49268 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:56.264862 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e70d7c7-c612-450b-8738-78c57c54848a None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:56.265825 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e70d7c7-c612-450b-8738-78c57c54848a None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:56.266940 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e70d7c7-c612-450b-8738-78c57c54848a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:56.285969 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d295e8a0-e9af-40fc-aedb-420ebbcbba08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:56.286606 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295e8a0-e9af-40fc-aedb-420ebbcbba08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:56.288810 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295e8a0-e9af-40fc-aedb-420ebbcbba08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:56.288810 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295e8a0-e9af-40fc-aedb-420ebbcbba08 admin admin] PATH_INFO: `/v3/users/43402622442e436a9b19ca130b8d91ce` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:56.289004 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-8e70d7c7-c612-450b-8738-78c57c54848a None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:56.289768 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d295e8a0-e9af-40fc-aedb-420ebbcbba08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43402622442e436a9b19ca130b8d91ce)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:56.290173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-8e70d7c7-c612-450b-8738-78c57c54848a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:56.291737 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d295e8a0-e9af-40fc-aedb-420ebbcbba08 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:56.298635 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1098/2205] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:27:56.308122 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b216a-6b5d-4ae9-b825-bdf4bb38c78f None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:56.308512 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b216a-6b5d-4ae9-b825-bdf4bb38c78f None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:56.308870 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b216a-6b5d-4ae9-b825-bdf4bb38c78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:56.323225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:56.342400 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-036b216a-6b5d-4ae9-b825-bdf4bb38c78f None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:56.343758 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-036b216a-6b5d-4ae9-b825-bdf4bb38c78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:56.350038 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d295e8a0-e9af-40fc-aedb-420ebbcbba08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43402622442e436a9b19ca130b8d91ce'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:56.351234 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d295e8a0-e9af-40fc-aedb-420ebbcbba08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43402622442e436a9b19ca130b8d91ce'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:56.351866 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d295e8a0-e9af-40fc-aedb-420ebbcbba08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43402622442e436a9b19ca130b8d91ce'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:56.353111 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1099/2206] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:27:56.357276 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b216a-6b5d-4ae9-b825-bdf4bb38c78f None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:56.367035 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d295e8a0-e9af-40fc-aedb-420ebbcbba08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43402622442e436a9b19ca130b8d91ce'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:56.367314 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b216a-6b5d-4ae9-b825-bdf4bb38c78f None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:56.372196 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4acf87-c4d0-4a9b-a75b-f049064eabb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:56.372798 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4acf87-c4d0-4a9b-a75b-f049064eabb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:56.373074 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4acf87-c4d0-4a9b-a75b-f049064eabb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:56.373348 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4acf87-c4d0-4a9b-a75b-f049064eabb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:56.373991 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4acf87-c4d0-4a9b-a75b-f049064eabb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:27:56.374288 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4acf87-c4d0-4a9b-a75b-f049064eabb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:56.375280 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d295e8a0-e9af-40fc-aedb-420ebbcbba08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43402622442e436a9b19ca130b8d91ce'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:56.376020 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4acf87-c4d0-4a9b-a75b-f049064eabb3 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:56.395078 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1100/2207] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:27:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:27:56.398644 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1108/2208] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:27:56 2022] DELETE /identity/v3/users/43402622442e436a9b19ca130b8d91ce => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:27:56.398786 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4acf87-c4d0-4a9b-a75b-f049064eabb3 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:56.403185 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-4f4acf87-c4d0-4a9b-a75b-f049064eabb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:56.419705 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4acf87-c4d0-4a9b-a75b-f049064eabb3 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:56.489736 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e427db-ce0f-46c4-84da-b1b0f2e18eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:56.490706 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e427db-ce0f-46c4-84da-b1b0f2e18eb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:56.492158 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e427db-ce0f-46c4-84da-b1b0f2e18eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:56.492585 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e427db-ce0f-46c4-84da-b1b0f2e18eb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:56.493446 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e427db-ce0f-46c4-84da-b1b0f2e18eb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:56.495697 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e427db-ce0f-46c4-84da-b1b0f2e18eb9 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:56.537439 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1101/2209] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:27:56 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:27:56.541225 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d295e8a0-e9af-40fc-aedb-420ebbcbba08 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:56.551399 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d295e8a0-e9af-40fc-aedb-420ebbcbba08 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:56.568325 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:56.620557 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d175f09-505d-49d3-9db2-c05692641c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:56.621195 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d175f09-505d-49d3-9db2-c05692641c59 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:56.621689 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d175f09-505d-49d3-9db2-c05692641c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:56.621967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d175f09-505d-49d3-9db2-c05692641c59 admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:56.622834 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d175f09-505d-49d3-9db2-c05692641c59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:56.625116 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d175f09-505d-49d3-9db2-c05692641c59 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:56.647650 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1109/2210] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:27:56 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:27:56.652146 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e427db-ce0f-46c4-84da-b1b0f2e18eb9 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:56.664286 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e427db-ce0f-46c4-84da-b1b0f2e18eb9 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:56.669319 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba5aec-5174-42dd-821f-241231e4923d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:56.669979 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba5aec-5174-42dd-821f-241231e4923d admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:56.670258 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba5aec-5174-42dd-821f-241231e4923d admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:56.670527 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba5aec-5174-42dd-821f-241231e4923d admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:56.671237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bba5aec-5174-42dd-821f-241231e4923d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:56.672853 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bba5aec-5174-42dd-821f-241231e4923d admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:56.683336 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1102/2211] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 14:27:56.687435 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d175f09-505d-49d3-9db2-c05692641c59 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:56.698752 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d175f09-505d-49d3-9db2-c05692641c59 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:56.706830 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2aea87-fda7-465d-aa8e-6cbb06f56d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:56.707462 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2aea87-fda7-465d-aa8e-6cbb06f56d3d admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:56.707740 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2aea87-fda7-465d-aa8e-6cbb06f56d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:56.708011 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2aea87-fda7-465d-aa8e-6cbb06f56d3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:56.710535 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2aea87-fda7-465d-aa8e-6cbb06f56d3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:56.711028 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2aea87-fda7-465d-aa8e-6cbb06f56d3d admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:56.723236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1110/2212] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 14:27:56.729467 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba5aec-5174-42dd-821f-241231e4923d admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:56.759131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba5aec-5174-42dd-821f-241231e4923d admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:56.783344 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6f0c7c-657d-4610-bc01-35186fd78274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:56.784030 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6f0c7c-657d-4610-bc01-35186fd78274 admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:56.784306 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6f0c7c-657d-4610-bc01-35186fd78274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:56.784575 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6f0c7c-657d-4610-bc01-35186fd78274 admin admin] PATH_INFO: `/v3/projects/2b331179ff764d15a454980577c9a765/users/feade913b8164701a35765804afdec79/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:56.803290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6f0c7c-657d-4610-bc01-35186fd78274 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b331179ff764d15a454980577c9a765, user_id=feade913b8164701a35765804afdec79, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:56.807463 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6f0c7c-657d-4610-bc01-35186fd78274 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:56.825852 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1103/2213] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:27:56 2022] PUT /identity/v3/projects/2b331179ff764d15a454980577c9a765/users/feade913b8164701a35765804afdec79/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:56.836984 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01008fb-0090-46e6-9fab-dae94eab7215 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:56.837292 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01008fb-0090-46e6-9fab-dae94eab7215 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:56.837541 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01008fb-0090-46e6-9fab-dae94eab7215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:56.874578 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-e01008fb-0090-46e6-9fab-dae94eab7215 None None] MFA Rules not processed for user `feade913b8164701a35765804afdec79`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:56.877232 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-e01008fb-0090-46e6-9fab-dae94eab7215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:56.975615 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1111/2214] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:27:56.977004 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6f0c7c-657d-4610-bc01-35186fd78274 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:57.098321 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6f0c7c-657d-4610-bc01-35186fd78274 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:57.113029 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f768a1a-851b-4504-bbf0-5db4c68f1056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:57.113678 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f768a1a-851b-4504-bbf0-5db4c68f1056 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:57.113945 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f768a1a-851b-4504-bbf0-5db4c68f1056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:57.114236 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f768a1a-851b-4504-bbf0-5db4c68f1056 admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:57.115147 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f768a1a-851b-4504-bbf0-5db4c68f1056 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:57.120528 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f768a1a-851b-4504-bbf0-5db4c68f1056 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:57.181756 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1104/2215] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:27:56 2022] POST /identity/v3/projects => generated 385 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:27:57.185486 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f768a1a-851b-4504-bbf0-5db4c68f1056 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:57.196674 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01008fb-0090-46e6-9fab-dae94eab7215 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:57.199291 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-e01008fb-0090-46e6-9fab-dae94eab7215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:57.237930 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01008fb-0090-46e6-9fab-dae94eab7215 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:57.251043 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:57.364240 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f768a1a-851b-4504-bbf0-5db4c68f1056 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:57.379050 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ec173-f68c-4056-9ed8-1643730de1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:57.379050 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ec173-f68c-4056-9ed8-1643730de1b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:57.379050 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ec173-f68c-4056-9ed8-1643730de1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:57.379050 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ec173-f68c-4056-9ed8-1643730de1b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:57.379518 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6ec173-f68c-4056-9ed8-1643730de1b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:57.379518 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6ec173-f68c-4056-9ed8-1643730de1b7 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:57.416750 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a284a2-7541-4bcc-ba94-417c62cd0ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:57.417577 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a284a2-7541-4bcc-ba94-417c62cd0ce1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:57.417970 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a284a2-7541-4bcc-ba94-417c62cd0ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:57.418340 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a284a2-7541-4bcc-ba94-417c62cd0ce1 admin admin] PATH_INFO: `/v3/projects/3f37e42e01664a5eb97e7b3f52b8c6f9` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:57.419680 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a284a2-7541-4bcc-ba94-417c62cd0ce1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f37e42e01664a5eb97e7b3f52b8c6f9)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:57.421335 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1105/2216] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:27:57 2022] POST /identity/v3/users => generated 293 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:27:57.424133 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a284a2-7541-4bcc-ba94-417c62cd0ce1 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:57.425175 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ec173-f68c-4056-9ed8-1643730de1b7 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:57.453707 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ec173-f68c-4056-9ed8-1643730de1b7 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:57.468868 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278e173-8f0f-4599-83a4-0c9e475385cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:57.469929 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278e173-8f0f-4599-83a4-0c9e475385cd admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:57.470170 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278e173-8f0f-4599-83a4-0c9e475385cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:57.470436 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278e173-8f0f-4599-83a4-0c9e475385cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:57.471878 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d278e173-8f0f-4599-83a4-0c9e475385cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:57.474840 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d278e173-8f0f-4599-83a4-0c9e475385cd admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:57.499505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1106/2217] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:27:57.501900 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278e173-8f0f-4599-83a4-0c9e475385cd admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:57.516705 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:57.524417 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278e173-8f0f-4599-83a4-0c9e475385cd admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:57.557597 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d4a284a2-7541-4bcc-ba94-417c62cd0ce1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f37e42e01664a5eb97e7b3f52b8c6f9', 'request_id': 'req-d4a284a2-7541-4bcc-ba94-417c62cd0ce1'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:57.566108 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d4a284a2-7541-4bcc-ba94-417c62cd0ce1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f37e42e01664a5eb97e7b3f52b8c6f9', 'request_id': 'req-d4a284a2-7541-4bcc-ba94-417c62cd0ce1'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:57.575673 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d4a284a2-7541-4bcc-ba94-417c62cd0ce1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f37e42e01664a5eb97e7b3f52b8c6f9', 'request_id': 'req-d4a284a2-7541-4bcc-ba94-417c62cd0ce1'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:57.601111 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d4a284a2-7541-4bcc-ba94-417c62cd0ce1 admin admin] The token cache is being invalidate because project 3f37e42e01664a5eb97e7b3f52b8c6f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:27:57.605348 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d4a284a2-7541-4bcc-ba94-417c62cd0ce1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:57.605654 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d4a284a2-7541-4bcc-ba94-417c62cd0ce1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:57.607176 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1112/2218] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:27:57 2022] DELETE /identity/v3/projects/3f37e42e01664a5eb97e7b3f52b8c6f9 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:57.622319 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a284a2-7541-4bcc-ba94-417c62cd0ce1 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:57.624346 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-d4a284a2-7541-4bcc-ba94-417c62cd0ce1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:57.650375 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b2d6ce-3d23-44d3-a679-766cb21bd7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:57.651266 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2d6ce-3d23-44d3-a679-766cb21bd7fe admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:57.651623 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2d6ce-3d23-44d3-a679-766cb21bd7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:57.651969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2d6ce-3d23-44d3-a679-766cb21bd7fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:57.652694 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b2d6ce-3d23-44d3-a679-766cb21bd7fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:57.654387 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b2d6ce-3d23-44d3-a679-766cb21bd7fe admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:57.668531 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1107/2219] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:27:57.669555 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b2d6ce-3d23-44d3-a679-766cb21bd7fe admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:57.671721 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-35b2d6ce-3d23-44d3-a679-766cb21bd7fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:57.690868 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b2d6ce-3d23-44d3-a679-766cb21bd7fe admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:57.697671 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31233cec-d8e4-495a-981f-e95d39e482a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:57.698339 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31233cec-d8e4-495a-981f-e95d39e482a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:57.698693 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31233cec-d8e4-495a-981f-e95d39e482a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:57.698917 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31233cec-d8e4-495a-981f-e95d39e482a0 admin admin] PATH_INFO: `/v3/projects/0c3c00fe35f44060a1e3e9b292b3a742/users/38dfee1ed69f4fe29c0f045c6cadaa78/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:57.716548 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31233cec-d8e4-495a-981f-e95d39e482a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c3c00fe35f44060a1e3e9b292b3a742, user_id=38dfee1ed69f4fe29c0f045c6cadaa78, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:57.718348 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a284a2-7541-4bcc-ba94-417c62cd0ce1 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:57.723621 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5e7904-eb2e-4366-9281-87f4c222918f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:57.724262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e7904-eb2e-4366-9281-87f4c222918f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:57.724536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e7904-eb2e-4366-9281-87f4c222918f admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:57.724811 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e7904-eb2e-4366-9281-87f4c222918f admin admin] PATH_INFO: `/v3/users/f8af518ff36c4c9ab9d965b1e6242358` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:57.726632 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5e7904-eb2e-4366-9281-87f4c222918f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8af518ff36c4c9ab9d965b1e6242358)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:57.726974 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31233cec-d8e4-495a-981f-e95d39e482a0 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:57.733746 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5e7904-eb2e-4366-9281-87f4c222918f admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:57.747519 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1108/2220] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:27:57 2022] PUT /identity/v3/projects/0c3c00fe35f44060a1e3e9b292b3a742/users/38dfee1ed69f4fe29c0f045c6cadaa78/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:27:57.760655 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26235df6-f905-4fb9-b198-7ce680a4b0cf None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:57.761023 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26235df6-f905-4fb9-b198-7ce680a4b0cf None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:57.761389 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26235df6-f905-4fb9-b198-7ce680a4b0cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:57.802591 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-26235df6-f905-4fb9-b198-7ce680a4b0cf None None] MFA Rules not processed for user `38dfee1ed69f4fe29c0f045c6cadaa78`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:57.804201 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9d5e7904-eb2e-4366-9281-87f4c222918f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8af518ff36c4c9ab9d965b1e6242358'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:57.804356 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-26235df6-f905-4fb9-b198-7ce680a4b0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:57.805105 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9d5e7904-eb2e-4366-9281-87f4c222918f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8af518ff36c4c9ab9d965b1e6242358'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:57.805845 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9d5e7904-eb2e-4366-9281-87f4c222918f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8af518ff36c4c9ab9d965b1e6242358'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:57.829405 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9d5e7904-eb2e-4366-9281-87f4c222918f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8af518ff36c4c9ab9d965b1e6242358'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:57.843042 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9d5e7904-eb2e-4366-9281-87f4c222918f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8af518ff36c4c9ab9d965b1e6242358'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:57.869509 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1113/2221] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:27:57 2022] DELETE /identity/v3/users/f8af518ff36c4c9ab9d965b1e6242358 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:27:57.898539 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:57.905993 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1109/2222] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:27:57.911511 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5e7904-eb2e-4366-9281-87f4c222918f admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:57.913040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-9d5e7904-eb2e-4366-9281-87f4c222918f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:57.980146 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:58.005968 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5e7904-eb2e-4366-9281-87f4c222918f admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:58.011146 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1cbfe5-a516-48d8-93d6-b6265327fc5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:58.011780 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cbfe5-a516-48d8-93d6-b6265327fc5e admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:58.012060 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cbfe5-a516-48d8-93d6-b6265327fc5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:58.012349 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cbfe5-a516-48d8-93d6-b6265327fc5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:58.012979 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1cbfe5-a516-48d8-93d6-b6265327fc5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:58.015248 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1cbfe5-a516-48d8-93d6-b6265327fc5e admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:58.051764 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1114/2223] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:27:57 2022] POST /identity/v3/projects => generated 383 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:27:58.056523 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26235df6-f905-4fb9-b198-7ce680a4b0cf None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:58.144449 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26235df6-f905-4fb9-b198-7ce680a4b0cf None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:58.207318 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1cbfe5-a516-48d8-93d6-b6265327fc5e admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:58.207318 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-ed1cbfe5-a516-48d8-93d6-b6265327fc5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:58.230222 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ed25e-ecb8-493c-9408-52ecf937de2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:58.231440 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ed25e-ecb8-493c-9408-52ecf937de2b admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:58.231721 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ed25e-ecb8-493c-9408-52ecf937de2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:58.231990 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ed25e-ecb8-493c-9408-52ecf937de2b admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:58.232643 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6ed25e-ecb8-493c-9408-52ecf937de2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:58.234577 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6ed25e-ecb8-493c-9408-52ecf937de2b admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:58.260547 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1110/2224] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:27:58 2022] POST /identity/v3/users => generated 298 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:27:58.263968 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ed25e-ecb8-493c-9408-52ecf937de2b admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:58.275981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ed25e-ecb8-493c-9408-52ecf937de2b admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:58.280965 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630dda00-6c8d-480f-a9d1-060eeb6b59a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:58.281467 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630dda00-6c8d-480f-a9d1-060eeb6b59a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:58.281745 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630dda00-6c8d-480f-a9d1-060eeb6b59a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:58.282020 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630dda00-6c8d-480f-a9d1-060eeb6b59a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:58.282968 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630dda00-6c8d-480f-a9d1-060eeb6b59a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:58.284587 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630dda00-6c8d-480f-a9d1-060eeb6b59a2 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:58.286115 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1cbfe5-a516-48d8-93d6-b6265327fc5e admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:58.291894 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe4532a-eca0-4247-a40a-e157dc3fe6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:58.292692 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe4532a-eca0-4247-a40a-e157dc3fe6bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:58.292970 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe4532a-eca0-4247-a40a-e157dc3fe6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:58.293263 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe4532a-eca0-4247-a40a-e157dc3fe6bf admin admin] PATH_INFO: `/v3/projects/44c7d33290854b6ab828c86c2c4c10b2` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:58.294389 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe4532a-eca0-4247-a40a-e157dc3fe6bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=44c7d33290854b6ab828c86c2c4c10b2)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:58.295352 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1111/2225] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 14:27:58.296324 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe4532a-eca0-4247-a40a-e157dc3fe6bf admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:58.299372 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630dda00-6c8d-480f-a9d1-060eeb6b59a2 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:58.309445 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630dda00-6c8d-480f-a9d1-060eeb6b59a2 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:58.314293 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b9cf4-6a54-47d6-bebb-a9a3af11856e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:58.315002 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b9cf4-6a54-47d6-bebb-a9a3af11856e admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:58.315227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b9cf4-6a54-47d6-bebb-a9a3af11856e admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:58.315497 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b9cf4-6a54-47d6-bebb-a9a3af11856e admin admin] PATH_INFO: `/v3/projects/0886ed557a1d4fc1bf0744c313454ee4/users/83ec4ff2f9854c529ddf32410156abc9/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:58.337709 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81b9cf4-6a54-47d6-bebb-a9a3af11856e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0886ed557a1d4fc1bf0744c313454ee4, user_id=83ec4ff2f9854c529ddf32410156abc9, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:58.340609 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81b9cf4-6a54-47d6-bebb-a9a3af11856e admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:58.356207 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-cfe4532a-eca0-4247-a40a-e157dc3fe6bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44c7d33290854b6ab828c86c2c4c10b2', 'request_id': 'req-cfe4532a-eca0-4247-a40a-e157dc3fe6bf'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:58.359351 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1112/2226] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:27:58 2022] PUT /identity/v3/projects/0886ed557a1d4fc1bf0744c313454ee4/users/83ec4ff2f9854c529ddf32410156abc9/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:27:58.362720 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-cfe4532a-eca0-4247-a40a-e157dc3fe6bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44c7d33290854b6ab828c86c2c4c10b2', 'request_id': 'req-cfe4532a-eca0-4247-a40a-e157dc3fe6bf'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:58.363436 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b9cf4-6a54-47d6-bebb-a9a3af11856e admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:58.369328 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-cfe4532a-eca0-4247-a40a-e157dc3fe6bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44c7d33290854b6ab828c86c2c4c10b2', 'request_id': 'req-cfe4532a-eca0-4247-a40a-e157dc3fe6bf'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:58.382980 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-cfe4532a-eca0-4247-a40a-e157dc3fe6bf admin admin] The token cache is being invalidate because project 44c7d33290854b6ab828c86c2c4c10b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:27:58.383347 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-cfe4532a-eca0-4247-a40a-e157dc3fe6bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:58.384486 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-cfe4532a-eca0-4247-a40a-e157dc3fe6bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:27:58.386505 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1115/2227] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:27:58 2022] DELETE /identity/v3/projects/44c7d33290854b6ab828c86c2c4c10b2 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:27:58.394897 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe29914-89fb-4ddc-bb8c-0df974770262 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:58.395270 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe29914-89fb-4ddc-bb8c-0df974770262 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:58.395594 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe29914-89fb-4ddc-bb8c-0df974770262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:58.424225 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-cbe29914-89fb-4ddc-bb8c-0df974770262 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:58.425873 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-cbe29914-89fb-4ddc-bb8c-0df974770262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:58.451816 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b9cf4-6a54-47d6-bebb-a9a3af11856e admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:58.463614 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884418ca-62a5-4b95-b961-042973591470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:58.464461 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884418ca-62a5-4b95-b961-042973591470 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:58.465389 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884418ca-62a5-4b95-b961-042973591470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:58.465389 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884418ca-62a5-4b95-b961-042973591470 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:58.466077 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884418ca-62a5-4b95-b961-042973591470 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:58.467776 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884418ca-62a5-4b95-b961-042973591470 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:58.469419 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:58.481423 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1113/2228] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 14:27:58.485528 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884418ca-62a5-4b95-b961-042973591470 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:58.487713 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-884418ca-62a5-4b95-b961-042973591470 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:58.488387 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:58.515033 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884418ca-62a5-4b95-b961-042973591470 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:58.515147 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1116/2229] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:27:58.515147 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59934ce2-f6cd-4cc7-92f3-e6a078f70345 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:58.515147 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59934ce2-f6cd-4cc7-92f3-e6a078f70345 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:58.515377 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cff0d4-830b-4a40-b211-ea5673287e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:58.515491 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59934ce2-f6cd-4cc7-92f3-e6a078f70345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:58.515583 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cff0d4-830b-4a40-b211-ea5673287e0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:58.515583 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cff0d4-830b-4a40-b211-ea5673287e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:58.515583 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cff0d4-830b-4a40-b211-ea5673287e0a admin admin] PATH_INFO: `/v3/projects/0886ed557a1d4fc1bf0744c313454ee4/users/83ec4ff2f9854c529ddf32410156abc9/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:58.515583 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cff0d4-830b-4a40-b211-ea5673287e0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0886ed557a1d4fc1bf0744c313454ee4, user_id=83ec4ff2f9854c529ddf32410156abc9, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:58.519085 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cff0d4-830b-4a40-b211-ea5673287e0a admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:58.528774 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1114/2230] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:27:58 2022] PUT /identity/v3/projects/0886ed557a1d4fc1bf0744c313454ee4/users/83ec4ff2f9854c529ddf32410156abc9/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:27:58.540318 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a9343-c2c7-4cbe-a443-745a7ac69d70 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:58.540633 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a9343-c2c7-4cbe-a443-745a7ac69d70 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:58.540871 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a9343-c2c7-4cbe-a443-745a7ac69d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:58.546466 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-59934ce2-f6cd-4cc7-92f3-e6a078f70345 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:58.548140 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-59934ce2-f6cd-4cc7-92f3-e6a078f70345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:58.570712 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-661a9343-c2c7-4cbe-a443-745a7ac69d70 None None] MFA Rules not processed for user `83ec4ff2f9854c529ddf32410156abc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:58.571880 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-661a9343-c2c7-4cbe-a443-745a7ac69d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:58.647862 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1117/2231] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:27:58.649922 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59934ce2-f6cd-4cc7-92f3-e6a078f70345 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:58.667370 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59934ce2-f6cd-4cc7-92f3-e6a078f70345 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:58.673240 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73479cf1-87e4-4622-82a6-5cc1bec9c705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:58.675182 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73479cf1-87e4-4622-82a6-5cc1bec9c705 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:58.676049 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73479cf1-87e4-4622-82a6-5cc1bec9c705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:58.679624 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73479cf1-87e4-4622-82a6-5cc1bec9c705 admin admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:58.680843 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73479cf1-87e4-4622-82a6-5cc1bec9c705 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:27:58.681677 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73479cf1-87e4-4622-82a6-5cc1bec9c705 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:58.691961 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73479cf1-87e4-4622-82a6-5cc1bec9c705 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:58.696573 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1115/2232] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:58 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:27:58.706016 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1118/2233] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:27:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:27:58.710718 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661a9343-c2c7-4cbe-a443-745a7ac69d70 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:58.712171 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73479cf1-87e4-4622-82a6-5cc1bec9c705 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:58.714226 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-73479cf1-87e4-4622-82a6-5cc1bec9c705 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:58.724180 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661a9343-c2c7-4cbe-a443-745a7ac69d70 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:58.730088 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5299f312-8fef-49d2-821f-f92f3ff1ea26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:58.730797 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5299f312-8fef-49d2-821f-f92f3ff1ea26 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:58.731179 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5299f312-8fef-49d2-821f-f92f3ff1ea26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:58.731521 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5299f312-8fef-49d2-821f-f92f3ff1ea26 admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:58.732287 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5299f312-8fef-49d2-821f-f92f3ff1ea26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:58.737012 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5299f312-8fef-49d2-821f-f92f3ff1ea26 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:58.782576 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1116/2234] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:27:58 2022] POST /identity/v3/projects => generated 367 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:27:58.787065 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5299f312-8fef-49d2-821f-f92f3ff1ea26 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:58.809259 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73479cf1-87e4-4622-82a6-5cc1bec9c705 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:58.883681 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a9b737-d96c-44d9-98ca-88be0d65aaae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:58.884327 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9b737-d96c-44d9-98ca-88be0d65aaae service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:58.884749 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9b737-d96c-44d9-98ca-88be0d65aaae service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:58.885033 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9b737-d96c-44d9-98ca-88be0d65aaae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:58.886855 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-61a9b737-d96c-44d9-98ca-88be0d65aaae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:58.887080 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5299f312-8fef-49d2-821f-f92f3ff1ea26 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:58.899930 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259570e-2b32-457f-abaf-4d6c419a9cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:58.901048 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259570e-2b32-457f-abaf-4d6c419a9cf4 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:58.901685 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259570e-2b32-457f-abaf-4d6c419a9cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:58.905868 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259570e-2b32-457f-abaf-4d6c419a9cf4 admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:58.905868 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c259570e-2b32-457f-abaf-4d6c419a9cf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:58.909920 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c259570e-2b32-457f-abaf-4d6c419a9cf4 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:58.938700 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1117/2235] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:27:58 2022] POST /identity/v3/users => generated 284 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:27:58.939733 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259570e-2b32-457f-abaf-4d6c419a9cf4 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:58.949326 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259570e-2b32-457f-abaf-4d6c419a9cf4 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:58.954579 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:58.955171 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa8c8e1-1bb4-4e6b-b17f-bedf5487b819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:58.955760 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8c8e1-1bb4-4e6b-b17f-bedf5487b819 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:58.956098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8c8e1-1bb4-4e6b-b17f-bedf5487b819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:58.956316 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8c8e1-1bb4-4e6b-b17f-bedf5487b819 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:58.956926 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa8c8e1-1bb4-4e6b-b17f-bedf5487b819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:58.958505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa8c8e1-1bb4-4e6b-b17f-bedf5487b819 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:58.967257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a9b737-d96c-44d9-98ca-88be0d65aaae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:58.968693 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a9b737-d96c-44d9-98ca-88be0d65aaae service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:58.969697 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1118/2236] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:27:58.973623 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa8c8e1-1bb4-4e6b-b17f-bedf5487b819 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:58.979269 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1119/2237] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:27:58 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:27:58.984358 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa8c8e1-1bb4-4e6b-b17f-bedf5487b819 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:58.989606 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de8f85b-9578-4bbf-ab06-791f5a9297f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:58.990202 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8f85b-9578-4bbf-ab06-791f5a9297f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:58.990478 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8f85b-9578-4bbf-ab06-791f5a9297f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:58.990778 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8f85b-9578-4bbf-ab06-791f5a9297f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:58.991451 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de8f85b-9578-4bbf-ab06-791f5a9297f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:58.993051 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de8f85b-9578-4bbf-ab06-791f5a9297f1 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:59.004064 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1119/2238] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 14:27:59.008182 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a9b737-d96c-44d9-98ca-88be0d65aaae service cinder] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:59.019481 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a9b737-d96c-44d9-98ca-88be0d65aaae service cinder] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:59.024870 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad63e2e-d8d0-4fd5-afb8-ce436599586e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:59.025760 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad63e2e-d8d0-4fd5-afb8-ce436599586e admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:59.026044 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad63e2e-d8d0-4fd5-afb8-ce436599586e admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:59.026429 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad63e2e-d8d0-4fd5-afb8-ce436599586e admin admin] PATH_INFO: `/v3/projects/bdefe36206d04b8fba5175d870303eb0/users/36866a1362aa4c8aa21d86132eb51c42/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:59.042959 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad63e2e-d8d0-4fd5-afb8-ce436599586e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdefe36206d04b8fba5175d870303eb0, user_id=36866a1362aa4c8aa21d86132eb51c42, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:59.045718 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad63e2e-d8d0-4fd5-afb8-ce436599586e admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:59.062878 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1120/2239] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:27:59 2022] PUT /identity/v3/projects/bdefe36206d04b8fba5175d870303eb0/users/36866a1362aa4c8aa21d86132eb51c42/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:27:59.072677 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc32fc7-1b3f-4a30-b841-a19601ca6bf8 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:59.072963 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc32fc7-1b3f-4a30-b841-a19601ca6bf8 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:59.073236 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc32fc7-1b3f-4a30-b841-a19601ca6bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:59.104705 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:59.108443 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-6cc32fc7-1b3f-4a30-b841-a19601ca6bf8 None None] MFA Rules not processed for user `36866a1362aa4c8aa21d86132eb51c42`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:59.110165 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-6cc32fc7-1b3f-4a30-b841-a19601ca6bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:59.202726 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1120/2240] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:27:59.207332 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad63e2e-d8d0-4fd5-afb8-ce436599586e admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:59.292001 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad63e2e-d8d0-4fd5-afb8-ce436599586e admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:59.298442 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d5ef0-28e0-44ac-8698-44efcb290188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:59.299902 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d5ef0-28e0-44ac-8698-44efcb290188 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:59.300355 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d5ef0-28e0-44ac-8698-44efcb290188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:59.300811 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d5ef0-28e0-44ac-8698-44efcb290188 admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:59.301728 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870d5ef0-28e0-44ac-8698-44efcb290188 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:59.303854 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870d5ef0-28e0-44ac-8698-44efcb290188 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:59.351454 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1121/2241] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:27:59 2022] POST /identity/v3/projects => generated 367 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:27:59.355577 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc32fc7-1b3f-4a30-b841-a19601ca6bf8 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:59.405839 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc32fc7-1b3f-4a30-b841-a19601ca6bf8 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:59.502215 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd6a8d9-b60c-4a99-8c28-b1ebcc429359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:59.502927 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6a8d9-b60c-4a99-8c28-b1ebcc429359 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:59.503274 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6a8d9-b60c-4a99-8c28-b1ebcc429359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:59.503551 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6a8d9-b60c-4a99-8c28-b1ebcc429359 admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:59.504282 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd6a8d9-b60c-4a99-8c28-b1ebcc429359 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:59.506292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd6a8d9-b60c-4a99-8c28-b1ebcc429359 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:59.534072 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1121/2242] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:27:59 2022] POST /identity/v3/users => generated 290 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:27:59.535547 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d5ef0-28e0-44ac-8698-44efcb290188 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:59.547983 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d5ef0-28e0-44ac-8698-44efcb290188 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:59.553851 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d712e8-0393-4f66-8ce7-7356a3095aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:59.554472 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d712e8-0393-4f66-8ce7-7356a3095aa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:59.554754 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d712e8-0393-4f66-8ce7-7356a3095aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:59.555053 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d712e8-0393-4f66-8ce7-7356a3095aa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:59.555698 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d712e8-0393-4f66-8ce7-7356a3095aa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:59.557298 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d712e8-0393-4f66-8ce7-7356a3095aa6 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:59.570858 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1122/2243] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 14:27:59.573691 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd6a8d9-b60c-4a99-8c28-b1ebcc429359 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:59.583864 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd6a8d9-b60c-4a99-8c28-b1ebcc429359 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:59.588552 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a18f2-4662-4502-bd09-9b5c775f8aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:59.589264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a18f2-4662-4502-bd09-9b5c775f8aff admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:59.589631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a18f2-4662-4502-bd09-9b5c775f8aff admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:59.590022 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a18f2-4662-4502-bd09-9b5c775f8aff admin admin] PATH_INFO: `/v3/projects/1220aab914ec4469a4b2b229b2c55309/users/a7f8d9f25c8a40e492408fd6476f985a/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:59.606893 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131a18f2-4662-4502-bd09-9b5c775f8aff admin admin] RBAC: Authorizing `identity:create_grant(project_id=1220aab914ec4469a4b2b229b2c55309, user_id=a7f8d9f25c8a40e492408fd6476f985a, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:59.610593 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131a18f2-4662-4502-bd09-9b5c775f8aff admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:59.625086 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1122/2244] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:27:59 2022] PUT /identity/v3/projects/1220aab914ec4469a4b2b229b2c55309/users/a7f8d9f25c8a40e492408fd6476f985a/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:27:59.629159 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d712e8-0393-4f66-8ce7-7356a3095aa6 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:59.639966 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d712e8-0393-4f66-8ce7-7356a3095aa6 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:59.704291 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb86804-88f4-4223-8020-ac64a477a388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:59.705023 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb86804-88f4-4223-8020-ac64a477a388 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:59.705395 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb86804-88f4-4223-8020-ac64a477a388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:59.705763 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb86804-88f4-4223-8020-ac64a477a388 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:59.706523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb86804-88f4-4223-8020-ac64a477a388 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:59.708415 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb86804-88f4-4223-8020-ac64a477a388 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:59.719680 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1123/2245] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:27:59 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:27:59.723808 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a18f2-4662-4502-bd09-9b5c775f8aff admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:27:59.734335 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a18f2-4662-4502-bd09-9b5c775f8aff admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:27:59.739123 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ba27d-5008-42dc-b6bd-40cc35add58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:27:59.746623 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ba27d-5008-42dc-b6bd-40cc35add58e admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:59.746623 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ba27d-5008-42dc-b6bd-40cc35add58e admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:59.746623 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ba27d-5008-42dc-b6bd-40cc35add58e admin admin] PATH_INFO: `/v3/projects/1220aab914ec4469a4b2b229b2c55309/users/a7f8d9f25c8a40e492408fd6476f985a/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:59.752695 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03ba27d-5008-42dc-b6bd-40cc35add58e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1220aab914ec4469a4b2b229b2c55309, user_id=a7f8d9f25c8a40e492408fd6476f985a, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:27:59.755810 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03ba27d-5008-42dc-b6bd-40cc35add58e admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:27:59.768718 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1123/2246] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:27:59 2022] PUT /identity/v3/projects/1220aab914ec4469a4b2b229b2c55309/users/a7f8d9f25c8a40e492408fd6476f985a/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:27:59.780292 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa48d0-c480-4d4e-91f4-8f19e2b35d29 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:27:59.780647 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa48d0-c480-4d4e-91f4-8f19e2b35d29 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:27:59.780989 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa48d0-c480-4d4e-91f4-8f19e2b35d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:27:59.810520 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-cfaa48d0-c480-4d4e-91f4-8f19e2b35d29 None None] MFA Rules not processed for user `a7f8d9f25c8a40e492408fd6476f985a`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:27:59.811769 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-cfaa48d0-c480-4d4e-91f4-8f19e2b35d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:27:59.846056 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:27:59.901634 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1124/2247] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:27:59 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:27:59.913907 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ba27d-5008-42dc-b6bd-40cc35add58e admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:00.000490 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ba27d-5008-42dc-b6bd-40cc35add58e admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:00.006584 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e8ffe-6146-456d-bd8c-15c05082277f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:00.007597 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e8ffe-6146-456d-bd8c-15c05082277f service cinder] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:00.007920 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e8ffe-6146-456d-bd8c-15c05082277f service cinder] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:00.008191 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e8ffe-6146-456d-bd8c-15c05082277f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:00.026914 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78e8ffe-6146-456d-bd8c-15c05082277f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:00.026914 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78e8ffe-6146-456d-bd8c-15c05082277f service cinder] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:00.028804 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:00.039430 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1124/2248] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:27:59 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 14:28:00.238746 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa48d0-c480-4d4e-91f4-8f19e2b35d29 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:00.255131 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa48d0-c480-4d4e-91f4-8f19e2b35d29 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:00.259414 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c4cd89-dbc3-43ba-9cff-a958cb594d6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:00.260228 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4cd89-dbc3-43ba-9cff-a958cb594d6e service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:00.260513 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4cd89-dbc3-43ba-9cff-a958cb594d6e service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:00.261095 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4cd89-dbc3-43ba-9cff-a958cb594d6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:00.336486 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c4cd89-dbc3-43ba-9cff-a958cb594d6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:00.338060 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c4cd89-dbc3-43ba-9cff-a958cb594d6e service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:00.347585 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1125/2249] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:28:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:28:01.024714 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e8ffe-6146-456d-bd8c-15c05082277f service cinder] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:01.034577 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e8ffe-6146-456d-bd8c-15c05082277f service cinder] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:01.038728 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ea97b-1e77-4824-bdca-0aa61bbae0df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:01.039357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ea97b-1e77-4824-bdca-0aa61bbae0df service cinder] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:01.039638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ea97b-1e77-4824-bdca-0aa61bbae0df service cinder] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:01.039909 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ea97b-1e77-4824-bdca-0aa61bbae0df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:01.041717 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-be3ea97b-1e77-4824-bdca-0aa61bbae0df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:01.104451 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ea97b-1e77-4824-bdca-0aa61bbae0df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:01.105889 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ea97b-1e77-4824-bdca-0aa61bbae0df service cinder] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:01.115017 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1125/2250] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:28:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:28:01.807107 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c4cd89-dbc3-43ba-9cff-a958cb594d6e service cinder] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:01.815909 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c4cd89-dbc3-43ba-9cff-a958cb594d6e service cinder] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:01.819885 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c181f24f-983c-4173-b97d-777770db700b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:01.820495 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181f24f-983c-4173-b97d-777770db700b service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:01.820775 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181f24f-983c-4173-b97d-777770db700b service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:01.821052 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181f24f-983c-4173-b97d-777770db700b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:01.906180 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c181f24f-983c-4173-b97d-777770db700b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:01.908378 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c181f24f-983c-4173-b97d-777770db700b service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:01.920110 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1126/2251] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:28:01 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:28:05.697155 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ea97b-1e77-4824-bdca-0aa61bbae0df service cinder] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:05.843195 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ea97b-1e77-4824-bdca-0aa61bbae0df service cinder] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:05.858175 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7311a-f39c-48ef-9456-0063be31365a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:05.858918 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7311a-f39c-48ef-9456-0063be31365a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:05.858918 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7311a-f39c-48ef-9456-0063be31365a admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:05.859489 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7311a-f39c-48ef-9456-0063be31365a admin admin] PATH_INFO: `/v3/users/36866a1362aa4c8aa21d86132eb51c42` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:05.862213 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7311a-f39c-48ef-9456-0063be31365a admin admin] RBAC: Authorizing `identity:delete_user(user_id=36866a1362aa4c8aa21d86132eb51c42)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:05.863306 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7311a-f39c-48ef-9456-0063be31365a admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:05.994096 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-42c7311a-f39c-48ef-9456-0063be31365a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36866a1362aa4c8aa21d86132eb51c42'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:05.996195 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-42c7311a-f39c-48ef-9456-0063be31365a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36866a1362aa4c8aa21d86132eb51c42'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:06.011643 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-42c7311a-f39c-48ef-9456-0063be31365a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36866a1362aa4c8aa21d86132eb51c42'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:06.012809 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-42c7311a-f39c-48ef-9456-0063be31365a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36866a1362aa4c8aa21d86132eb51c42'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:06.041680 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-42c7311a-f39c-48ef-9456-0063be31365a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36866a1362aa4c8aa21d86132eb51c42'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:06.060596 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1126/2252] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:28:05 2022] DELETE /identity/v3/users/36866a1362aa4c8aa21d86132eb51c42 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:06.065902 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070bbd4-3770-4fcd-ad1d-add27c82b932 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:06.066649 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070bbd4-3770-4fcd-ad1d-add27c82b932 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:06.066649 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070bbd4-3770-4fcd-ad1d-add27c82b932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:06.095345 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-6070bbd4-3770-4fcd-ad1d-add27c82b932 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:06.096788 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-6070bbd4-3770-4fcd-ad1d-add27c82b932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:06.164713 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:06.176679 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1127/2253] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:06.187096 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7311a-f39c-48ef-9456-0063be31365a admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:06.189187 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-42c7311a-f39c-48ef-9456-0063be31365a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:06.207499 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:06.255812 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7311a-f39c-48ef-9456-0063be31365a admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:06.261205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f132c5-0fef-4c5b-af76-b3191699dfbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:06.261824 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f132c5-0fef-4c5b-af76-b3191699dfbd service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:06.262107 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f132c5-0fef-4c5b-af76-b3191699dfbd service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:06.262377 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f132c5-0fef-4c5b-af76-b3191699dfbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:06.276909 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f132c5-0fef-4c5b-af76-b3191699dfbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:06.278680 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f132c5-0fef-4c5b-af76-b3191699dfbd service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:06.290122 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1127/2254] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:28:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:28:06.583642 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070bbd4-3770-4fcd-ad1d-add27c82b932 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:06.585760 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-6070bbd4-3770-4fcd-ad1d-add27c82b932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:06.602578 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070bbd4-3770-4fcd-ad1d-add27c82b932 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:06.636243 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c4686-ed48-44a0-98be-f8c7c745260e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:06.637146 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c4686-ed48-44a0-98be-f8c7c745260e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:06.637968 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c4686-ed48-44a0-98be-f8c7c745260e admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:06.638321 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c4686-ed48-44a0-98be-f8c7c745260e admin admin] PATH_INFO: `/v3/projects/bdefe36206d04b8fba5175d870303eb0` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:06.639674 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5c4686-ed48-44a0-98be-f8c7c745260e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdefe36206d04b8fba5175d870303eb0)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:06.641642 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5c4686-ed48-44a0-98be-f8c7c745260e admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:06.692747 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-be5c4686-ed48-44a0-98be-f8c7c745260e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdefe36206d04b8fba5175d870303eb0', 'request_id': 'req-be5c4686-ed48-44a0-98be-f8c7c745260e'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:06.699582 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-be5c4686-ed48-44a0-98be-f8c7c745260e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdefe36206d04b8fba5175d870303eb0', 'request_id': 'req-be5c4686-ed48-44a0-98be-f8c7c745260e'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:06.705220 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-be5c4686-ed48-44a0-98be-f8c7c745260e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdefe36206d04b8fba5175d870303eb0', 'request_id': 'req-be5c4686-ed48-44a0-98be-f8c7c745260e'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:06.718435 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-be5c4686-ed48-44a0-98be-f8c7c745260e admin admin] The token cache is being invalidate because project bdefe36206d04b8fba5175d870303eb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:28:06.718776 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-be5c4686-ed48-44a0-98be-f8c7c745260e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:06.719368 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-be5c4686-ed48-44a0-98be-f8c7c745260e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:06.721046 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1128/2255] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:28:06 2022] DELETE /identity/v3/projects/bdefe36206d04b8fba5175d870303eb0 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:06.725088 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f132c5-0fef-4c5b-af76-b3191699dfbd service neutron] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:06.736437 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f132c5-0fef-4c5b-af76-b3191699dfbd service neutron] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:06.825707 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1416d9-3f63-42d5-bbe2-4cd9d0a9d844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:06.826365 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1416d9-3f63-42d5-bbe2-4cd9d0a9d844 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:06.826646 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1416d9-3f63-42d5-bbe2-4cd9d0a9d844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:06.826946 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1416d9-3f63-42d5-bbe2-4cd9d0a9d844 admin admin] PATH_INFO: `/v3/users/a7f8d9f25c8a40e492408fd6476f985a` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:06.828109 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1416d9-3f63-42d5-bbe2-4cd9d0a9d844 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7f8d9f25c8a40e492408fd6476f985a)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:06.830139 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1416d9-3f63-42d5-bbe2-4cd9d0a9d844 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:06.899842 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c4686-ed48-44a0-98be-f8c7c745260e admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:06.902112 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-be5c4686-ed48-44a0-98be-f8c7c745260e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:06.909902 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-ee1416d9-3f63-42d5-bbe2-4cd9d0a9d844 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7f8d9f25c8a40e492408fd6476f985a'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:06.911135 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-ee1416d9-3f63-42d5-bbe2-4cd9d0a9d844 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7f8d9f25c8a40e492408fd6476f985a'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:06.920505 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c4686-ed48-44a0-98be-f8c7c745260e admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:06.922664 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-ee1416d9-3f63-42d5-bbe2-4cd9d0a9d844 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7f8d9f25c8a40e492408fd6476f985a'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:06.923972 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-ee1416d9-3f63-42d5-bbe2-4cd9d0a9d844 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7f8d9f25c8a40e492408fd6476f985a'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:06.928008 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0051e3c7-c2c6-4b4e-827a-c9fc93114f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:06.929091 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0051e3c7-c2c6-4b4e-827a-c9fc93114f75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:06.930694 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0051e3c7-c2c6-4b4e-827a-c9fc93114f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:06.931396 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0051e3c7-c2c6-4b4e-827a-c9fc93114f75 admin admin] PATH_INFO: `/v3/users/777ba964e39048f2a7f6e00349a9e465` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:06.937745 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0051e3c7-c2c6-4b4e-827a-c9fc93114f75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=777ba964e39048f2a7f6e00349a9e465)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:06.940306 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0051e3c7-c2c6-4b4e-827a-c9fc93114f75 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:06.947804 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-ee1416d9-3f63-42d5-bbe2-4cd9d0a9d844 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7f8d9f25c8a40e492408fd6476f985a'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:06.993206 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1128/2256] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:28:06 2022] DELETE /identity/v3/users/a7f8d9f25c8a40e492408fd6476f985a => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:07.029747 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-0051e3c7-c2c6-4b4e-827a-c9fc93114f75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '777ba964e39048f2a7f6e00349a9e465'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:07.030555 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-0051e3c7-c2c6-4b4e-827a-c9fc93114f75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '777ba964e39048f2a7f6e00349a9e465'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:07.031399 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-0051e3c7-c2c6-4b4e-827a-c9fc93114f75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '777ba964e39048f2a7f6e00349a9e465'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:07.049374 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-0051e3c7-c2c6-4b4e-827a-c9fc93114f75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '777ba964e39048f2a7f6e00349a9e465'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:07.076160 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-0051e3c7-c2c6-4b4e-827a-c9fc93114f75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '777ba964e39048f2a7f6e00349a9e465'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:07.110389 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1129/2257] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:28:06 2022] DELETE /identity/v3/users/777ba964e39048f2a7f6e00349a9e465 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:28:07.113313 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3571a1-4aa0-44af-b492-2f9c70c694c0 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:07.113908 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3571a1-4aa0-44af-b492-2f9c70c694c0 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:07.114381 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3571a1-4aa0-44af-b492-2f9c70c694c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:07.143188 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-5d3571a1-4aa0-44af-b492-2f9c70c694c0 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:07.143675 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-5d3571a1-4aa0-44af-b492-2f9c70c694c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:07.253777 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:07.258732 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0051e3c7-c2c6-4b4e-827a-c9fc93114f75 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:07.261484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-0051e3c7-c2c6-4b4e-827a-c9fc93114f75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:07.340359 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1129/2258] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:07.359004 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3571a1-4aa0-44af-b492-2f9c70c694c0 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:07.360453 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-5d3571a1-4aa0-44af-b492-2f9c70c694c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:07.375152 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:07.400728 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0051e3c7-c2c6-4b4e-827a-c9fc93114f75 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:07.406767 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba0b65-f338-4b0b-914b-ecbb1915b0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:07.411020 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba0b65-f338-4b0b-914b-ecbb1915b0c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:07.413534 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba0b65-f338-4b0b-914b-ecbb1915b0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:07.414715 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba0b65-f338-4b0b-914b-ecbb1915b0c0 admin admin] PATH_INFO: `/v3/users/feade913b8164701a35765804afdec79` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:07.416153 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eba0b65-f338-4b0b-914b-ecbb1915b0c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=feade913b8164701a35765804afdec79)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:07.418444 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eba0b65-f338-4b0b-914b-ecbb1915b0c0 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:07.480972 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3571a1-4aa0-44af-b492-2f9c70c694c0 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:07.487458 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adba390-86ef-42d2-aedb-55fe52cd44e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:07.488409 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adba390-86ef-42d2-aedb-55fe52cd44e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:07.488757 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adba390-86ef-42d2-aedb-55fe52cd44e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:07.489097 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adba390-86ef-42d2-aedb-55fe52cd44e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:07.503922 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adba390-86ef-42d2-aedb-55fe52cd44e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:07.507178 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adba390-86ef-42d2-aedb-55fe52cd44e5 service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:07.519858 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9eba0b65-f338-4b0b-914b-ecbb1915b0c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'feade913b8164701a35765804afdec79'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:07.521216 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9eba0b65-f338-4b0b-914b-ecbb1915b0c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'feade913b8164701a35765804afdec79'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:07.522406 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9eba0b65-f338-4b0b-914b-ecbb1915b0c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'feade913b8164701a35765804afdec79'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:07.532949 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1130/2259] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:28:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 14:28:07.551089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9eba0b65-f338-4b0b-914b-ecbb1915b0c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'feade913b8164701a35765804afdec79'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:07.563439 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9eba0b65-f338-4b0b-914b-ecbb1915b0c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'feade913b8164701a35765804afdec79'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:07.588881 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1130/2260] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:28:07 2022] DELETE /identity/v3/users/feade913b8164701a35765804afdec79 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:07.593696 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184dc1b8-ff56-43c5-9204-401cb3c5d341 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:07.593975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184dc1b8-ff56-43c5-9204-401cb3c5d341 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:07.594237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184dc1b8-ff56-43c5-9204-401cb3c5d341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:07.632566 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-184dc1b8-ff56-43c5-9204-401cb3c5d341 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:07.634326 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-184dc1b8-ff56-43c5-9204-401cb3c5d341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:07.710173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1131/2261] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:07.735344 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba0b65-f338-4b0b-914b-ecbb1915b0c0 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:07.736900 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-9eba0b65-f338-4b0b-914b-ecbb1915b0c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:07.838566 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba0b65-f338-4b0b-914b-ecbb1915b0c0 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:07.848494 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e376d31b-406d-433d-95ad-8020a6615112 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:07.849285 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376d31b-406d-433d-95ad-8020a6615112 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:07.849616 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376d31b-406d-433d-95ad-8020a6615112 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:07.849937 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376d31b-406d-433d-95ad-8020a6615112 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:07.859885 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e376d31b-406d-433d-95ad-8020a6615112 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:07.861324 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e376d31b-406d-433d-95ad-8020a6615112 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:07.873810 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1131/2262] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:28:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:28:07.994316 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184dc1b8-ff56-43c5-9204-401cb3c5d341 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:07.996105 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-184dc1b8-ff56-43c5-9204-401cb3c5d341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:08.005438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e376d31b-406d-433d-95ad-8020a6615112 service neutron] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:08.008745 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-e376d31b-406d-433d-95ad-8020a6615112 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:08.011817 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184dc1b8-ff56-43c5-9204-401cb3c5d341 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:08.017339 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1840b34d-85a4-48c4-9723-17069bb60295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:08.017990 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840b34d-85a4-48c4-9723-17069bb60295 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:08.018453 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840b34d-85a4-48c4-9723-17069bb60295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:08.031859 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e376d31b-406d-433d-95ad-8020a6615112 service neutron] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:08.039492 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f868f0ac-d64d-4a74-ba1b-cc49bcd7bb46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:08.040312 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868f0ac-d64d-4a74-ba1b-cc49bcd7bb46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:08.040817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868f0ac-d64d-4a74-ba1b-cc49bcd7bb46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:08.041515 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868f0ac-d64d-4a74-ba1b-cc49bcd7bb46 admin admin] PATH_INFO: `/v3/projects/1220aab914ec4469a4b2b229b2c55309` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:08.041956 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840b34d-85a4-48c4-9723-17069bb60295 admin admin] PATH_INFO: `/v3/projects/2b916ad697234d9c9562efe5d9e111c7` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:08.044663 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f868f0ac-d64d-4a74-ba1b-cc49bcd7bb46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1220aab914ec4469a4b2b229b2c55309)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:08.047458 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f868f0ac-d64d-4a74-ba1b-cc49bcd7bb46 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:08.048865 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1840b34d-85a4-48c4-9723-17069bb60295 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b916ad697234d9c9562efe5d9e111c7)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:08.057995 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1840b34d-85a4-48c4-9723-17069bb60295 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:08.132679 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-f868f0ac-d64d-4a74-ba1b-cc49bcd7bb46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1220aab914ec4469a4b2b229b2c55309', 'request_id': 'req-f868f0ac-d64d-4a74-ba1b-cc49bcd7bb46'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:08.140021 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-f868f0ac-d64d-4a74-ba1b-cc49bcd7bb46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1220aab914ec4469a4b2b229b2c55309', 'request_id': 'req-f868f0ac-d64d-4a74-ba1b-cc49bcd7bb46'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:08.147987 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-f868f0ac-d64d-4a74-ba1b-cc49bcd7bb46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1220aab914ec4469a4b2b229b2c55309', 'request_id': 'req-f868f0ac-d64d-4a74-ba1b-cc49bcd7bb46'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:08.154239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:08.156482 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-1840b34d-85a4-48c4-9723-17069bb60295 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b916ad697234d9c9562efe5d9e111c7', 'request_id': 'req-1840b34d-85a4-48c4-9723-17069bb60295'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:08.165973 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-1840b34d-85a4-48c4-9723-17069bb60295 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b916ad697234d9c9562efe5d9e111c7', 'request_id': 'req-1840b34d-85a4-48c4-9723-17069bb60295'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:08.174915 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-1840b34d-85a4-48c4-9723-17069bb60295 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b916ad697234d9c9562efe5d9e111c7', 'request_id': 'req-1840b34d-85a4-48c4-9723-17069bb60295'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:08.185744 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-f868f0ac-d64d-4a74-ba1b-cc49bcd7bb46 admin admin] The token cache is being invalidate because project 1220aab914ec4469a4b2b229b2c55309 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:28:08.186148 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-f868f0ac-d64d-4a74-ba1b-cc49bcd7bb46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:08.187173 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-f868f0ac-d64d-4a74-ba1b-cc49bcd7bb46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:08.199290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-1840b34d-85a4-48c4-9723-17069bb60295 admin admin] The token cache is being invalidate because project 2b916ad697234d9c9562efe5d9e111c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:28:08.200019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-1840b34d-85a4-48c4-9723-17069bb60295 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:08.200334 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-1840b34d-85a4-48c4-9723-17069bb60295 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:08.202350 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1132/2263] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:28:07 2022] DELETE /identity/v3/projects/2b916ad697234d9c9562efe5d9e111c7 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:08.203217 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1840b34d-85a4-48c4-9723-17069bb60295 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:08.204590 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-1840b34d-85a4-48c4-9723-17069bb60295 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:08.205695 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1132/2264] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:28:08 2022] DELETE /identity/v3/projects/1220aab914ec4469a4b2b229b2c55309 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:08.210437 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe90f1c-7bd0-4e20-b796-d53c15497ce7 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:08.210437 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe90f1c-7bd0-4e20-b796-d53c15497ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:08.210437 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe90f1c-7bd0-4e20-b796-d53c15497ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:08.221439 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:08.259099 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-0fe90f1c-7bd0-4e20-b796-d53c15497ce7 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:08.260227 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-0fe90f1c-7bd0-4e20-b796-d53c15497ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:08.306213 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1840b34d-85a4-48c4-9723-17069bb60295 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:08.311056 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa8ee99-5c65-4426-9024-19f9d3088553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:08.311694 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa8ee99-5c65-4426-9024-19f9d3088553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:08.311994 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa8ee99-5c65-4426-9024-19f9d3088553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:08.312227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa8ee99-5c65-4426-9024-19f9d3088553 admin admin] PATH_INFO: `/v3/users/75575c10a8eb4ebfab5b2f0c330541df` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:08.313363 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa8ee99-5c65-4426-9024-19f9d3088553 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75575c10a8eb4ebfab5b2f0c330541df)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:08.315347 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa8ee99-5c65-4426-9024-19f9d3088553 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:08.335549 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1133/2265] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:08.337364 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8348a31-bf3d-4b5f-a3c7-718bae5cd895 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:08.338104 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8348a31-bf3d-4b5f-a3c7-718bae5cd895 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:08.338874 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8348a31-bf3d-4b5f-a3c7-718bae5cd895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:08.363416 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-e8348a31-bf3d-4b5f-a3c7-718bae5cd895 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:08.364727 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-e8348a31-bf3d-4b5f-a3c7-718bae5cd895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:08.374264 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1134/2266] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:08.378458 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe976b1-6696-4cc9-b9f3-a61f7a6e7c98 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:08.380589 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-aaa8ee99-5c65-4426-9024-19f9d3088553 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75575c10a8eb4ebfab5b2f0c330541df'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:08.381413 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-aaa8ee99-5c65-4426-9024-19f9d3088553 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75575c10a8eb4ebfab5b2f0c330541df'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:08.383371 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe976b1-6696-4cc9-b9f3-a61f7a6e7c98 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:08.386223 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe976b1-6696-4cc9-b9f3-a61f7a6e7c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:08.392206 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-aaa8ee99-5c65-4426-9024-19f9d3088553 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75575c10a8eb4ebfab5b2f0c330541df'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:08.393060 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-aaa8ee99-5c65-4426-9024-19f9d3088553 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75575c10a8eb4ebfab5b2f0c330541df'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:08.417682 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-cbe976b1-6696-4cc9-b9f3-a61f7a6e7c98 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:08.417852 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-aaa8ee99-5c65-4426-9024-19f9d3088553 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75575c10a8eb4ebfab5b2f0c330541df'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:08.419807 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-cbe976b1-6696-4cc9-b9f3-a61f7a6e7c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:08.435949 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1133/2267] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:28:08 2022] DELETE /identity/v3/users/75575c10a8eb4ebfab5b2f0c330541df => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:08.437211 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2cc49-c170-40cf-a1d9-6dff19036d7b None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:08.437501 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2cc49-c170-40cf-a1d9-6dff19036d7b None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:08.437778 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2cc49-c170-40cf-a1d9-6dff19036d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:08.457842 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1135/2268] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:08.458707 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe976b1-6696-4cc9-b9f3-a61f7a6e7c98 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:08.458991 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-13f2cc49-c170-40cf-a1d9-6dff19036d7b None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:08.460771 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-13f2cc49-c170-40cf-a1d9-6dff19036d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:08.461218 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-cbe976b1-6696-4cc9-b9f3-a61f7a6e7c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:08.474503 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe976b1-6696-4cc9-b9f3-a61f7a6e7c98 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:08.546640 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e4f129-b709-426a-87f1-2d28f3697e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:08.547392 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4f129-b709-426a-87f1-2d28f3697e3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:08.547674 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4f129-b709-426a-87f1-2d28f3697e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:08.547945 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4f129-b709-426a-87f1-2d28f3697e3b admin admin] PATH_INFO: `/v3/projects/2b331179ff764d15a454980577c9a765` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:08.549369 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e4f129-b709-426a-87f1-2d28f3697e3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b331179ff764d15a454980577c9a765)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:08.551405 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e4f129-b709-426a-87f1-2d28f3697e3b admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:08.555619 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1134/2269] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:08.556558 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f2cc49-c170-40cf-a1d9-6dff19036d7b None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:08.567561 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f2cc49-c170-40cf-a1d9-6dff19036d7b None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:08.574275 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa6e8e-9512-4c63-b4d5-0da3c074cf4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:08.574926 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa6e8e-9512-4c63-b4d5-0da3c074cf4c admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:08.575366 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa6e8e-9512-4c63-b4d5-0da3c074cf4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:08.575771 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa6e8e-9512-4c63-b4d5-0da3c074cf4c admin admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:08.576527 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa6e8e-9512-4c63-b4d5-0da3c074cf4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:28:08.576713 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa6e8e-9512-4c63-b4d5-0da3c074cf4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:08.578316 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa6e8e-9512-4c63-b4d5-0da3c074cf4c admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:08.591624 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1135/2270] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:28:08.592555 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa6e8e-9512-4c63-b4d5-0da3c074cf4c admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:08.606219 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa6e8e-9512-4c63-b4d5-0da3c074cf4c admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:08.612837 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-b8e4f129-b709-426a-87f1-2d28f3697e3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b331179ff764d15a454980577c9a765', 'request_id': 'req-b8e4f129-b709-426a-87f1-2d28f3697e3b'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:08.620782 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-b8e4f129-b709-426a-87f1-2d28f3697e3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b331179ff764d15a454980577c9a765', 'request_id': 'req-b8e4f129-b709-426a-87f1-2d28f3697e3b'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:08.621704 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:08.629398 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-b8e4f129-b709-426a-87f1-2d28f3697e3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b331179ff764d15a454980577c9a765', 'request_id': 'req-b8e4f129-b709-426a-87f1-2d28f3697e3b'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:08.648401 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-b8e4f129-b709-426a-87f1-2d28f3697e3b admin admin] The token cache is being invalidate because project 2b331179ff764d15a454980577c9a765 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:28:08.648705 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-b8e4f129-b709-426a-87f1-2d28f3697e3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:08.649467 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-b8e4f129-b709-426a-87f1-2d28f3697e3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:08.651045 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1136/2271] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:28:08 2022] DELETE /identity/v3/projects/2b331179ff764d15a454980577c9a765 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:08.651790 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e4f129-b709-426a-87f1-2d28f3697e3b admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:08.653196 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-b8e4f129-b709-426a-87f1-2d28f3697e3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:08.671866 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:08.687125 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba191b-7839-4113-b0e1-020c67d54ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:08.687852 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba191b-7839-4113-b0e1-020c67d54ce4 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:08.688272 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba191b-7839-4113-b0e1-020c67d54ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:08.688643 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba191b-7839-4113-b0e1-020c67d54ce4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:08.689425 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fba191b-7839-4113-b0e1-020c67d54ce4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:28:08.689772 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fba191b-7839-4113-b0e1-020c67d54ce4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:08.691587 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fba191b-7839-4113-b0e1-020c67d54ce4 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:08.705637 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1136/2272] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:28:08.707893 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba191b-7839-4113-b0e1-020c67d54ce4 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:08.710409 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-4fba191b-7839-4113-b0e1-020c67d54ce4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:08.723349 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba191b-7839-4113-b0e1-020c67d54ce4 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:08.728645 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5bf237-f399-4a52-b9a8-ca4d81ddd52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:08.729262 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5bf237-f399-4a52-b9a8-ca4d81ddd52d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:08.729536 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5bf237-f399-4a52-b9a8-ca4d81ddd52d admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:08.729805 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5bf237-f399-4a52-b9a8-ca4d81ddd52d admin admin] PATH_INFO: `/v3/users/38dfee1ed69f4fe29c0f045c6cadaa78` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:08.730936 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5bf237-f399-4a52-b9a8-ca4d81ddd52d admin admin] RBAC: Authorizing `identity:delete_user(user_id=38dfee1ed69f4fe29c0f045c6cadaa78)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:08.733013 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5bf237-f399-4a52-b9a8-ca4d81ddd52d admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:08.738118 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e4f129-b709-426a-87f1-2d28f3697e3b admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:08.742764 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b848db-4a97-4c96-9ced-7351c60c1192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:08.743404 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b848db-4a97-4c96-9ced-7351c60c1192 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:08.743688 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b848db-4a97-4c96-9ced-7351c60c1192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:08.743957 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b848db-4a97-4c96-9ced-7351c60c1192 admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:08.744581 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b848db-4a97-4c96-9ced-7351c60c1192 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:08.746448 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b848db-4a97-4c96-9ced-7351c60c1192 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:08.780896 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1137/2273] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:28:08 2022] POST /identity/v3/projects => generated 399 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:28:08.781754 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b848db-4a97-4c96-9ced-7351c60c1192 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:08.783701 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-85b848db-4a97-4c96-9ced-7351c60c1192 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:08.784101 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-3f5bf237-f399-4a52-b9a8-ca4d81ddd52d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38dfee1ed69f4fe29c0f045c6cadaa78'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:08.784878 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-3f5bf237-f399-4a52-b9a8-ca4d81ddd52d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38dfee1ed69f4fe29c0f045c6cadaa78'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:08.792657 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-3f5bf237-f399-4a52-b9a8-ca4d81ddd52d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38dfee1ed69f4fe29c0f045c6cadaa78'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:08.793390 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-3f5bf237-f399-4a52-b9a8-ca4d81ddd52d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38dfee1ed69f4fe29c0f045c6cadaa78'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:08.805601 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-3f5bf237-f399-4a52-b9a8-ca4d81ddd52d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38dfee1ed69f4fe29c0f045c6cadaa78'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:08.827825 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1137/2274] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:28:08 2022] DELETE /identity/v3/users/38dfee1ed69f4fe29c0f045c6cadaa78 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:08.828844 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5bf237-f399-4a52-b9a8-ca4d81ddd52d admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:08.830351 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-3f5bf237-f399-4a52-b9a8-ca4d81ddd52d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:08.861655 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b848db-4a97-4c96-9ced-7351c60c1192 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:08.921263 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5bf237-f399-4a52-b9a8-ca4d81ddd52d admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:08.927121 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b08cd5-9bb6-4f14-bf21-93b2344ed592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:08.927740 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b08cd5-9bb6-4f14-bf21-93b2344ed592 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:08.928407 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b08cd5-9bb6-4f14-bf21-93b2344ed592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:08.928407 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b08cd5-9bb6-4f14-bf21-93b2344ed592 admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:08.929384 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b08cd5-9bb6-4f14-bf21-93b2344ed592 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:08.930681 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749139db-988c-4530-8036-4d291b59a867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:08.931447 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749139db-988c-4530-8036-4d291b59a867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:08.931581 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b08cd5-9bb6-4f14-bf21-93b2344ed592 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:08.932232 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749139db-988c-4530-8036-4d291b59a867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:08.932232 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749139db-988c-4530-8036-4d291b59a867 admin admin] PATH_INFO: `/v3/projects/6bb2f863f5834cd4a47a3ed9feda7c02` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:08.933458 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749139db-988c-4530-8036-4d291b59a867 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bb2f863f5834cd4a47a3ed9feda7c02)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:08.935612 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749139db-988c-4530-8036-4d291b59a867 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:08.962549 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1138/2275] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:28:08 2022] POST /identity/v3/projects => generated 373 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:28:08.963245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b08cd5-9bb6-4f14-bf21-93b2344ed592 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:08.965533 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-81b08cd5-9bb6-4f14-bf21-93b2344ed592 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:08.970260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:08.998548 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:09.000977 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-749139db-988c-4530-8036-4d291b59a867 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bb2f863f5834cd4a47a3ed9feda7c02', 'request_id': 'req-749139db-988c-4530-8036-4d291b59a867'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:09.013350 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-749139db-988c-4530-8036-4d291b59a867 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bb2f863f5834cd4a47a3ed9feda7c02', 'request_id': 'req-749139db-988c-4530-8036-4d291b59a867'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:09.021676 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-749139db-988c-4530-8036-4d291b59a867 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bb2f863f5834cd4a47a3ed9feda7c02', 'request_id': 'req-749139db-988c-4530-8036-4d291b59a867'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:09.039681 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-749139db-988c-4530-8036-4d291b59a867 admin admin] The token cache is being invalidate because project 6bb2f863f5834cd4a47a3ed9feda7c02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:28:09.039998 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-749139db-988c-4530-8036-4d291b59a867 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:09.040694 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-749139db-988c-4530-8036-4d291b59a867 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:09.050671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1138/2276] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:28:08 2022] DELETE /identity/v3/projects/6bb2f863f5834cd4a47a3ed9feda7c02 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:09.051635 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b08cd5-9bb6-4f14-bf21-93b2344ed592 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:09.052099 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749139db-988c-4530-8036-4d291b59a867 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:09.053339 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-749139db-988c-4530-8036-4d291b59a867 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:09.126107 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758df926-b131-4905-b268-44d4f6ea2176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:09.126840 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758df926-b131-4905-b268-44d4f6ea2176 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:09.127369 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758df926-b131-4905-b268-44d4f6ea2176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:09.127800 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758df926-b131-4905-b268-44d4f6ea2176 admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:09.128657 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758df926-b131-4905-b268-44d4f6ea2176 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:09.130328 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749139db-988c-4530-8036-4d291b59a867 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:09.130709 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758df926-b131-4905-b268-44d4f6ea2176 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:09.136122 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b6dacd-e73d-49a3-a741-f853a3ac53a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:09.136825 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6dacd-e73d-49a3-a741-f853a3ac53a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:09.137159 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6dacd-e73d-49a3-a741-f853a3ac53a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:09.137437 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6dacd-e73d-49a3-a741-f853a3ac53a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:09.138137 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b6dacd-e73d-49a3-a741-f853a3ac53a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:09.140205 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b6dacd-e73d-49a3-a741-f853a3ac53a9 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:09.165612 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1139/2277] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:28:08 2022] POST /identity/v3/users => generated 300 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:28:09.167040 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15168ecb-0716-4dee-a333-309c66c06a45 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:09.167396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15168ecb-0716-4dee-a333-309c66c06a45 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:09.167749 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15168ecb-0716-4dee-a333-309c66c06a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:09.180187 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1139/2278] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:28:09 2022] POST /identity/v3/users => generated 287 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:28:09.181045 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b6dacd-e73d-49a3-a741-f853a3ac53a9 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:09.185087 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-64b6dacd-e73d-49a3-a741-f853a3ac53a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:09.196725 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-15168ecb-0716-4dee-a333-309c66c06a45 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:09.198107 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-15168ecb-0716-4dee-a333-309c66c06a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:09.200183 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b6dacd-e73d-49a3-a741-f853a3ac53a9 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:09.204807 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9f7a08-69b9-4f2b-9e2a-035bdfce82ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:09.205439 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f7a08-69b9-4f2b-9e2a-035bdfce82ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:09.205717 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f7a08-69b9-4f2b-9e2a-035bdfce82ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:09.205988 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f7a08-69b9-4f2b-9e2a-035bdfce82ca admin admin] PATH_INFO: `/v3/projects/0c3c00fe35f44060a1e3e9b292b3a742` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:09.206116 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1140/2279] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:09.207120 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15168ecb-0716-4dee-a333-309c66c06a45 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:09.208357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-15168ecb-0716-4dee-a333-309c66c06a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:09.211859 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9f7a08-69b9-4f2b-9e2a-035bdfce82ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c3c00fe35f44060a1e3e9b292b3a742)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:09.213909 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9f7a08-69b9-4f2b-9e2a-035bdfce82ca admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:09.220781 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15168ecb-0716-4dee-a333-309c66c06a45 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:09.225395 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd82323f-b0d3-484a-84f5-5d8d80b0035e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:09.225993 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd82323f-b0d3-484a-84f5-5d8d80b0035e admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:09.226274 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd82323f-b0d3-484a-84f5-5d8d80b0035e admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:09.226546 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd82323f-b0d3-484a-84f5-5d8d80b0035e admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:09.227228 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd82323f-b0d3-484a-84f5-5d8d80b0035e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:09.228818 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd82323f-b0d3-484a-84f5-5d8d80b0035e admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:09.243925 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1141/2280] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:28:09.244945 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd82323f-b0d3-484a-84f5-5d8d80b0035e admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:09.255784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd82323f-b0d3-484a-84f5-5d8d80b0035e admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:09.271940 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-ab9f7a08-69b9-4f2b-9e2a-035bdfce82ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c3c00fe35f44060a1e3e9b292b3a742', 'request_id': 'req-ab9f7a08-69b9-4f2b-9e2a-035bdfce82ca'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:09.286350 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-ab9f7a08-69b9-4f2b-9e2a-035bdfce82ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c3c00fe35f44060a1e3e9b292b3a742', 'request_id': 'req-ab9f7a08-69b9-4f2b-9e2a-035bdfce82ca'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:09.293056 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-ab9f7a08-69b9-4f2b-9e2a-035bdfce82ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c3c00fe35f44060a1e3e9b292b3a742', 'request_id': 'req-ab9f7a08-69b9-4f2b-9e2a-035bdfce82ca'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:09.313504 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-ab9f7a08-69b9-4f2b-9e2a-035bdfce82ca admin admin] The token cache is being invalidate because project 0c3c00fe35f44060a1e3e9b292b3a742 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:28:09.313800 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-ab9f7a08-69b9-4f2b-9e2a-035bdfce82ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:09.314539 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-ab9f7a08-69b9-4f2b-9e2a-035bdfce82ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:09.316299 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1140/2281] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:28:09 2022] DELETE /identity/v3/projects/0c3c00fe35f44060a1e3e9b292b3a742 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:09.317486 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6c2e6-fbf2-4b68-9cfe-0ca67b582143 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:09.317770 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6c2e6-fbf2-4b68-9cfe-0ca67b582143 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:09.318040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6c2e6-fbf2-4b68-9cfe-0ca67b582143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:09.326964 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b61b5c-313f-4bf4-8fa9-8b182ca7d8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:09.328765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b61b5c-313f-4bf4-8fa9-8b182ca7d8b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:09.329116 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b61b5c-313f-4bf4-8fa9-8b182ca7d8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:09.329605 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b61b5c-313f-4bf4-8fa9-8b182ca7d8b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:09.330330 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b61b5c-313f-4bf4-8fa9-8b182ca7d8b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:09.332094 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b61b5c-313f-4bf4-8fa9-8b182ca7d8b6 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:09.343674 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-74e6c2e6-fbf2-4b68-9cfe-0ca67b582143 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:09.344825 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1142/2282] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:28:09.345718 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b61b5c-313f-4bf4-8fa9-8b182ca7d8b6 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:09.346201 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-74e6c2e6-fbf2-4b68-9cfe-0ca67b582143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:09.352413 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-19b61b5c-313f-4bf4-8fa9-8b182ca7d8b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:09.367142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1141/2283] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:09.368154 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6c2e6-fbf2-4b68-9cfe-0ca67b582143 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:09.369486 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-74e6c2e6-fbf2-4b68-9cfe-0ca67b582143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:09.375930 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b61b5c-313f-4bf4-8fa9-8b182ca7d8b6 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:09.380797 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cad01a7-41db-4cfc-ac88-1cd1fd4b864c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:09.381975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad01a7-41db-4cfc-ac88-1cd1fd4b864c admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:09.382253 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad01a7-41db-4cfc-ac88-1cd1fd4b864c admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:09.382523 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad01a7-41db-4cfc-ac88-1cd1fd4b864c admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:09.383012 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6c2e6-fbf2-4b68-9cfe-0ca67b582143 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:09.383224 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cad01a7-41db-4cfc-ac88-1cd1fd4b864c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:09.384930 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cad01a7-41db-4cfc-ac88-1cd1fd4b864c admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:09.388093 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb7ae57-5fb2-403b-b299-a96dafce70d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:09.388727 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7ae57-5fb2-403b-b299-a96dafce70d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:09.389010 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7ae57-5fb2-403b-b299-a96dafce70d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:09.389287 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7ae57-5fb2-403b-b299-a96dafce70d0 admin admin] PATH_INFO: `/v3/users/83ec4ff2f9854c529ddf32410156abc9` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:09.390501 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb7ae57-5fb2-403b-b299-a96dafce70d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83ec4ff2f9854c529ddf32410156abc9)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:09.392656 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb7ae57-5fb2-403b-b299-a96dafce70d0 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:09.397554 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1143/2284] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:28:09.398457 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cad01a7-41db-4cfc-ac88-1cd1fd4b864c admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:09.400152 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-0cad01a7-41db-4cfc-ac88-1cd1fd4b864c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:09.414259 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cad01a7-41db-4cfc-ac88-1cd1fd4b864c admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:09.419679 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35370d1-3831-4fd7-9ce3-262d2a5d3430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:09.420455 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35370d1-3831-4fd7-9ce3-262d2a5d3430 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:09.420870 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35370d1-3831-4fd7-9ce3-262d2a5d3430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:09.421139 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35370d1-3831-4fd7-9ce3-262d2a5d3430 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:09.424229 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35370d1-3831-4fd7-9ce3-262d2a5d3430 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:09.426457 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35370d1-3831-4fd7-9ce3-262d2a5d3430 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:09.427631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:09.440638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1144/2285] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:28:09.441631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35370d1-3831-4fd7-9ce3-262d2a5d3430 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:09.453753 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35370d1-3831-4fd7-9ce3-262d2a5d3430 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:09.454456 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=80,keystone:DELETE=15,keystone:INSERT=5 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:09.459090 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986bf45a-8d38-4463-84bd-feeab0b47509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:09.459689 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986bf45a-8d38-4463-84bd-feeab0b47509 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:09.460037 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986bf45a-8d38-4463-84bd-feeab0b47509 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:09.460379 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986bf45a-8d38-4463-84bd-feeab0b47509 admin admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:09.461227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986bf45a-8d38-4463-84bd-feeab0b47509 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:28:09.461531 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986bf45a-8d38-4463-84bd-feeab0b47509 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:09.463218 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986bf45a-8d38-4463-84bd-feeab0b47509 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:09.465567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-bcb7ae57-5fb2-403b-b299-a96dafce70d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83ec4ff2f9854c529ddf32410156abc9'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:09.466798 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-bcb7ae57-5fb2-403b-b299-a96dafce70d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83ec4ff2f9854c529ddf32410156abc9'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:09.467765 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-bcb7ae57-5fb2-403b-b299-a96dafce70d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83ec4ff2f9854c529ddf32410156abc9'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:09.475711 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1145/2286] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:28:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:28:09.476505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986bf45a-8d38-4463-84bd-feeab0b47509 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:09.478415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-986bf45a-8d38-4463-84bd-feeab0b47509 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:09.487225 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-bcb7ae57-5fb2-403b-b299-a96dafce70d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83ec4ff2f9854c529ddf32410156abc9'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:09.497447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986bf45a-8d38-4463-84bd-feeab0b47509 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:09.501432 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e40400f-7a3d-4943-88df-bf4a9b4f1082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:09.501595 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-bcb7ae57-5fb2-403b-b299-a96dafce70d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83ec4ff2f9854c529ddf32410156abc9'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:09.502470 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40400f-7a3d-4943-88df-bf4a9b4f1082 admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:09.502750 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40400f-7a3d-4943-88df-bf4a9b4f1082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:09.503044 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40400f-7a3d-4943-88df-bf4a9b4f1082 admin admin] PATH_INFO: `/v3/projects/f4520940040348c38251ff5a9ac6999f/users/08a42da5af7549afab59e966b8da84fa/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:09.520034 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1142/2287] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:28:09 2022] DELETE /identity/v3/users/83ec4ff2f9854c529ddf32410156abc9 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:09.520939 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb7ae57-5fb2-403b-b299-a96dafce70d0 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:09.522418 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-bcb7ae57-5fb2-403b-b299-a96dafce70d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:09.524234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e40400f-7a3d-4943-88df-bf4a9b4f1082 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4520940040348c38251ff5a9ac6999f, user_id=08a42da5af7549afab59e966b8da84fa, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:09.526992 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e40400f-7a3d-4943-88df-bf4a9b4f1082 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:09.544027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1146/2288] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:28:09 2022] PUT /identity/v3/projects/f4520940040348c38251ff5a9ac6999f/users/08a42da5af7549afab59e966b8da84fa/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:09.545028 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e40400f-7a3d-4943-88df-bf4a9b4f1082 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:09.556728 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-8e40400f-7a3d-4943-88df-bf4a9b4f1082 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:09.596598 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb7ae57-5fb2-403b-b299-a96dafce70d0 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:09.649400 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e40400f-7a3d-4943-88df-bf4a9b4f1082 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:09.655257 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a2cdf4-43b3-4cb8-99c4-85567e11a719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:09.655405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2cdf4-43b3-4cb8-99c4-85567e11a719 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:09.655675 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2cdf4-43b3-4cb8-99c4-85567e11a719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:09.655951 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2cdf4-43b3-4cb8-99c4-85567e11a719 admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:09.656823 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a2cdf4-43b3-4cb8-99c4-85567e11a719 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:09.658801 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a2cdf4-43b3-4cb8-99c4-85567e11a719 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:09.675272 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5f398-adcc-4c7b-bc30-a60f495230d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:09.676577 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5f398-adcc-4c7b-bc30-a60f495230d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:09.676577 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5f398-adcc-4c7b-bc30-a60f495230d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:09.676862 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5f398-adcc-4c7b-bc30-a60f495230d3 admin admin] PATH_INFO: `/v3/projects/81daffe6f5db4bbf9e66cde38b13f8c0/users/3b976453181c41a98b387517b137cf4d/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:09.694254 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1147/2289] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:28:09 2022] POST /identity/v3/projects => generated 367 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:28:09.694254 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b00dd-4b03-48ab-b788-f85cc09f30cc None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:09.694254 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b00dd-4b03-48ab-b788-f85cc09f30cc None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:09.694254 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b00dd-4b03-48ab-b788-f85cc09f30cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:09.696170 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f5f398-adcc-4c7b-bc30-a60f495230d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81daffe6f5db4bbf9e66cde38b13f8c0, user_id=3b976453181c41a98b387517b137cf4d, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:09.699644 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f5f398-adcc-4c7b-bc30-a60f495230d3 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:09.734302 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1143/2290] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:28:09 2022] PUT /identity/v3/projects/81daffe6f5db4bbf9e66cde38b13f8c0/users/3b976453181c41a98b387517b137cf4d/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:09.734302 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5f398-adcc-4c7b-bc30-a60f495230d3 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:09.739182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-004b00dd-4b03-48ab-b788-f85cc09f30cc None None] MFA Rules not processed for user `08a42da5af7549afab59e966b8da84fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:09.741255 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-004b00dd-4b03-48ab-b788-f85cc09f30cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:09.776930 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:09.800514 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5f398-adcc-4c7b-bc30-a60f495230d3 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:09.806506 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c20707-d53f-498f-af8c-a4be48c6e19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:09.807283 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c20707-d53f-498f-af8c-a4be48c6e19b admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:09.807633 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c20707-d53f-498f-af8c-a4be48c6e19b admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:09.808395 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c20707-d53f-498f-af8c-a4be48c6e19b admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:09.808847 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c20707-d53f-498f-af8c-a4be48c6e19b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:09.810843 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c20707-d53f-498f-af8c-a4be48c6e19b admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:09.828440 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1148/2291] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:09 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:09.829584 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f0e02-5f44-47e9-a0bc-3236891a0e78 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:09.829945 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f0e02-5f44-47e9-a0bc-3236891a0e78 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:09.830276 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f0e02-5f44-47e9-a0bc-3236891a0e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:09.844449 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1144/2292] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:28:09 2022] POST /identity/v3/users => generated 284 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:28:09.849569 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c20707-d53f-498f-af8c-a4be48c6e19b admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:09.861980 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c20707-d53f-498f-af8c-a4be48c6e19b admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:09.868838 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea2f825-b574-4741-99c5-57d726fc7ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:09.869642 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea2f825-b574-4741-99c5-57d726fc7ca9 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:09.870082 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea2f825-b574-4741-99c5-57d726fc7ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:09.870359 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea2f825-b574-4741-99c5-57d726fc7ca9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:09.871041 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea2f825-b574-4741-99c5-57d726fc7ca9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:09.874563 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-fa5f0e02-5f44-47e9-a0bc-3236891a0e78 None None] MFA Rules not processed for user `3b976453181c41a98b387517b137cf4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:09.875875 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-fa5f0e02-5f44-47e9-a0bc-3236891a0e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:09.879919 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea2f825-b574-4741-99c5-57d726fc7ca9 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:09.892795 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1145/2293] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 14:28:09.893717 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea2f825-b574-4741-99c5-57d726fc7ca9 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:09.895486 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-0ea2f825-b574-4741-99c5-57d726fc7ca9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:09.907123 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:09.953129 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1149/2294] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:09.954055 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f0e02-5f44-47e9-a0bc-3236891a0e78 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:09.955757 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-fa5f0e02-5f44-47e9-a0bc-3236891a0e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:09.959344 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea2f825-b574-4741-99c5-57d726fc7ca9 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:09.967259 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa649701-fa46-4041-9e92-e26a22c4c920 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:09.967962 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa649701-fa46-4041-9e92-e26a22c4c920 service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:09.968165 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa649701-fa46-4041-9e92-e26a22c4c920 service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:09.968501 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa649701-fa46-4041-9e92-e26a22c4c920 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:09.979841 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f0e02-5f44-47e9-a0bc-3236891a0e78 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:09.981918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa649701-fa46-4041-9e92-e26a22c4c920 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:09.983387 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa649701-fa46-4041-9e92-e26a22c4c920 service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:09.985143 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1b3ba3-cd3d-4926-9004-d9eb4c26434b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:09.985777 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1b3ba3-cd3d-4926-9004-d9eb4c26434b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:09.986058 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1b3ba3-cd3d-4926-9004-d9eb4c26434b admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:09.986332 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1b3ba3-cd3d-4926-9004-d9eb4c26434b admin admin] PATH_INFO: `/v3/projects/0886ed557a1d4fc1bf0744c313454ee4` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:09.987480 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1b3ba3-cd3d-4926-9004-d9eb4c26434b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0886ed557a1d4fc1bf0744c313454ee4)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:09.989397 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1b3ba3-cd3d-4926-9004-d9eb4c26434b admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:09.992835 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1146/2295] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:28:09 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:28:09.993616 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa649701-fa46-4041-9e92-e26a22c4c920 service cinder] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:10.003536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa649701-fa46-4041-9e92-e26a22c4c920 service cinder] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:10.008166 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed2113-2727-4cd9-99f3-6214c16099c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:10.008751 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed2113-2727-4cd9-99f3-6214c16099c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:10.009022 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed2113-2727-4cd9-99f3-6214c16099c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:10.009311 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed2113-2727-4cd9-99f3-6214c16099c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:10.009930 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ed2113-2727-4cd9-99f3-6214c16099c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:10.011711 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ed2113-2727-4cd9-99f3-6214c16099c6 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:10.029048 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1147/2296] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:28:10.030038 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed2113-2727-4cd9-99f3-6214c16099c6 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:10.032440 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-b8ed2113-2727-4cd9-99f3-6214c16099c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:10.045969 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed2113-2727-4cd9-99f3-6214c16099c6 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:10.056709 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-be1b3ba3-cd3d-4926-9004-d9eb4c26434b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0886ed557a1d4fc1bf0744c313454ee4', 'request_id': 'req-be1b3ba3-cd3d-4926-9004-d9eb4c26434b'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:10.064289 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-be1b3ba3-cd3d-4926-9004-d9eb4c26434b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0886ed557a1d4fc1bf0744c313454ee4', 'request_id': 'req-be1b3ba3-cd3d-4926-9004-d9eb4c26434b'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:10.070268 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-be1b3ba3-cd3d-4926-9004-d9eb4c26434b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0886ed557a1d4fc1bf0744c313454ee4', 'request_id': 'req-be1b3ba3-cd3d-4926-9004-d9eb4c26434b'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:10.089358 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-be1b3ba3-cd3d-4926-9004-d9eb4c26434b admin admin] The token cache is being invalidate because project 0886ed557a1d4fc1bf0744c313454ee4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:28:10.089894 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-be1b3ba3-cd3d-4926-9004-d9eb4c26434b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:10.090770 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-be1b3ba3-cd3d-4926-9004-d9eb4c26434b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:10.093355 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1150/2297] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:28:09 2022] DELETE /identity/v3/projects/0886ed557a1d4fc1bf0744c313454ee4 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:10.094384 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1b3ba3-cd3d-4926-9004-d9eb4c26434b admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:10.096001 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-be1b3ba3-cd3d-4926-9004-d9eb4c26434b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:10.108933 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0f85b-903b-4b3d-b401-19da9064cefd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:10.110024 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0f85b-903b-4b3d-b401-19da9064cefd service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:10.110299 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0f85b-903b-4b3d-b401-19da9064cefd service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:10.110572 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0f85b-903b-4b3d-b401-19da9064cefd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:10.112545 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-e2a0f85b-903b-4b3d-b401-19da9064cefd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:10.182230 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1b3ba3-cd3d-4926-9004-d9eb4c26434b admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:10.186612 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a0f85b-903b-4b3d-b401-19da9064cefd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:10.188251 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a0f85b-903b-4b3d-b401-19da9064cefd service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:10.188457 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf99be-9f46-4698-b836-7bd9abc2a44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:10.189178 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf99be-9f46-4698-b836-7bd9abc2a44d admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:10.189464 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf99be-9f46-4698-b836-7bd9abc2a44d admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:10.189743 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf99be-9f46-4698-b836-7bd9abc2a44d admin admin] PATH_INFO: `/v3/projects/6bbceb11e4b346ab86747237c3f27bee/users/30a350860ec440d0866c9a51f9c5243e/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:10.198190 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1148/2298] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:28:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:28:10.199451 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa187bd-1fd0-45f3-85d9-860184359bea None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:10.199795 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa187bd-1fd0-45f3-85d9-860184359bea None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:10.200075 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa187bd-1fd0-45f3-85d9-860184359bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:10.223126 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-cfa187bd-1fd0-45f3-85d9-860184359bea None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:10.224341 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-cfa187bd-1fd0-45f3-85d9-860184359bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:10.226404 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adf99be-9f46-4698-b836-7bd9abc2a44d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bbceb11e4b346ab86747237c3f27bee, user_id=30a350860ec440d0866c9a51f9c5243e, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:10.230715 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adf99be-9f46-4698-b836-7bd9abc2a44d admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:10.233150 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1149/2299] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:10.233395 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa187bd-1fd0-45f3-85d9-860184359bea None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:10.234797 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-cfa187bd-1fd0-45f3-85d9-860184359bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:10.243515 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:10.249271 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa187bd-1fd0-45f3-85d9-860184359bea None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:10.249427 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1151/2300] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:28:10 2022] PUT /identity/v3/projects/6bbceb11e4b346ab86747237c3f27bee/users/30a350860ec440d0866c9a51f9c5243e/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:10.250631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abefb377-0c52-473e-8e25-611fa99f17be None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:10.250953 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abefb377-0c52-473e-8e25-611fa99f17be None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:10.251414 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abefb377-0c52-473e-8e25-611fa99f17be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:10.276777 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-abefb377-0c52-473e-8e25-611fa99f17be None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:10.279104 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-abefb377-0c52-473e-8e25-611fa99f17be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:10.322299 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:10.341411 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e4ca23-5b4c-4be9-a43d-3ea7f4b4b4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:10.342168 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4ca23-5b4c-4be9-a43d-3ea7f4b4b4c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:10.342448 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4ca23-5b4c-4be9-a43d-3ea7f4b4b4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:10.342716 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4ca23-5b4c-4be9-a43d-3ea7f4b4b4c7 admin admin] PATH_INFO: `/v3/users/3b976453181c41a98b387517b137cf4d` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:10.343901 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e4ca23-5b4c-4be9-a43d-3ea7f4b4b4c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b976453181c41a98b387517b137cf4d)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:10.346270 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e4ca23-5b4c-4be9-a43d-3ea7f4b4b4c7 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:10.372843 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1152/2301] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:10.372843 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b7477-4323-4bc8-b41c-692eda4b3719 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:10.372843 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b7477-4323-4bc8-b41c-692eda4b3719 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:10.372843 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b7477-4323-4bc8-b41c-692eda4b3719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:10.414333 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-ca6b7477-4323-4bc8-b41c-692eda4b3719 None None] MFA Rules not processed for user `30a350860ec440d0866c9a51f9c5243e`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:10.422901 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-ca6b7477-4323-4bc8-b41c-692eda4b3719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:10.429336 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-f1e4ca23-5b4c-4be9-a43d-3ea7f4b4b4c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b976453181c41a98b387517b137cf4d'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:10.430434 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-f1e4ca23-5b4c-4be9-a43d-3ea7f4b4b4c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b976453181c41a98b387517b137cf4d'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:10.431227 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-f1e4ca23-5b4c-4be9-a43d-3ea7f4b4b4c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b976453181c41a98b387517b137cf4d'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:10.445146 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-f1e4ca23-5b4c-4be9-a43d-3ea7f4b4b4c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b976453181c41a98b387517b137cf4d'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:10.455361 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-f1e4ca23-5b4c-4be9-a43d-3ea7f4b4b4c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b976453181c41a98b387517b137cf4d'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:10.494146 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1150/2302] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:28:10 2022] DELETE /identity/v3/users/3b976453181c41a98b387517b137cf4d => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:10.495585 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e4ca23-5b4c-4be9-a43d-3ea7f4b4b4c7 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:10.498043 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-f1e4ca23-5b4c-4be9-a43d-3ea7f4b4b4c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:10.522650 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1153/2303] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:10.524030 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751853fb-d6c3-4937-830b-49d3316089e6 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:10.524439 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751853fb-d6c3-4937-830b-49d3316089e6 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:10.524798 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751853fb-d6c3-4937-830b-49d3316089e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:10.588122 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-751853fb-d6c3-4937-830b-49d3316089e6 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:10.591925 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-751853fb-d6c3-4937-830b-49d3316089e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:10.608933 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e4ca23-5b4c-4be9-a43d-3ea7f4b4b4c7 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:10.618714 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1154/2304] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:10.619503 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751853fb-d6c3-4937-830b-49d3316089e6 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:10.626720 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9abde6-caf5-45c4-b6fe-f0fa31e4fbbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:10.627044 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-751853fb-d6c3-4937-830b-49d3316089e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:10.627932 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9abde6-caf5-45c4-b6fe-f0fa31e4fbbb admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:10.628426 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9abde6-caf5-45c4-b6fe-f0fa31e4fbbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:10.629243 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9abde6-caf5-45c4-b6fe-f0fa31e4fbbb admin admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:10.630193 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9abde6-caf5-45c4-b6fe-f0fa31e4fbbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:28:10.631231 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9abde6-caf5-45c4-b6fe-f0fa31e4fbbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:10.632447 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9abde6-caf5-45c4-b6fe-f0fa31e4fbbb admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:10.648438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1151/2305] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:28:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:28:10.649519 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9abde6-caf5-45c4-b6fe-f0fa31e4fbbb admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:10.653518 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-6e9abde6-caf5-45c4-b6fe-f0fa31e4fbbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:10.675588 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:10.702164 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751853fb-d6c3-4937-830b-49d3316089e6 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:10.709932 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce5a6e-4ecc-42a0-bb46-58a0b3528a1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:10.710611 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce5a6e-4ecc-42a0-bb46-58a0b3528a1f service cinder] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:10.711022 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce5a6e-4ecc-42a0-bb46-58a0b3528a1f service cinder] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:10.711384 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce5a6e-4ecc-42a0-bb46-58a0b3528a1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:10.724158 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9abde6-caf5-45c4-b6fe-f0fa31e4fbbb admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:10.727683 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e663afe5-3178-4975-b1c0-fc0271be9d16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:10.728285 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663afe5-3178-4975-b1c0-fc0271be9d16 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:10.728562 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663afe5-3178-4975-b1c0-fc0271be9d16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:10.728834 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663afe5-3178-4975-b1c0-fc0271be9d16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:10.737927 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e663afe5-3178-4975-b1c0-fc0271be9d16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:10.739393 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e663afe5-3178-4975-b1c0-fc0271be9d16 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:10.749780 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1152/2306] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:28:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:28:10.750580 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e663afe5-3178-4975-b1c0-fc0271be9d16 service neutron] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:10.760403 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e663afe5-3178-4975-b1c0-fc0271be9d16 service neutron] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:10.765863 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d0f39c-b1ac-4a88-8090-f1b623fd4496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:10.766455 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d0f39c-b1ac-4a88-8090-f1b623fd4496 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:10.766731 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d0f39c-b1ac-4a88-8090-f1b623fd4496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:10.767105 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d0f39c-b1ac-4a88-8090-f1b623fd4496 admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:10.767731 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d0f39c-b1ac-4a88-8090-f1b623fd4496 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:10.769613 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d0f39c-b1ac-4a88-8090-f1b623fd4496 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:10.776942 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ce5a6e-4ecc-42a0-bb46-58a0b3528a1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:10.778344 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ce5a6e-4ecc-42a0-bb46-58a0b3528a1f service cinder] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:10.789363 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1155/2307] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:28:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:28:10.810767 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1153/2308] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:28:10 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:28:10.811459 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce5a6e-4ecc-42a0-bb46-58a0b3528a1f service cinder] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:10.823071 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce5a6e-4ecc-42a0-bb46-58a0b3528a1f service cinder] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:10.827577 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:10.894379 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65071cc-ebc3-488f-a1d5-912ba50b6bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:10.895019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65071cc-ebc3-488f-a1d5-912ba50b6bf7 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:10.895296 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65071cc-ebc3-488f-a1d5-912ba50b6bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:10.895597 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65071cc-ebc3-488f-a1d5-912ba50b6bf7 admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:10.896227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65071cc-ebc3-488f-a1d5-912ba50b6bf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:10.898748 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65071cc-ebc3-488f-a1d5-912ba50b6bf7 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:10.923941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1156/2309] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:28:10 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:28:10.930703 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d0f39c-b1ac-4a88-8090-f1b623fd4496 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:10.949931 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d0f39c-b1ac-4a88-8090-f1b623fd4496 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:10.959528 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a682bd07-a9e8-4d27-b30b-60279ec262c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:10.960830 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682bd07-a9e8-4d27-b30b-60279ec262c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:10.961361 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682bd07-a9e8-4d27-b30b-60279ec262c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:10.961840 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682bd07-a9e8-4d27-b30b-60279ec262c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:10.963136 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a682bd07-a9e8-4d27-b30b-60279ec262c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:10.968191 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a682bd07-a9e8-4d27-b30b-60279ec262c7 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:10.994888 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1154/2310] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:28:10.998974 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65071cc-ebc3-488f-a1d5-912ba50b6bf7 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:11.010318 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65071cc-ebc3-488f-a1d5-912ba50b6bf7 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:11.015972 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88862c15-d498-487a-a972-63f905bcbc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:11.016674 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88862c15-d498-487a-a972-63f905bcbc3d admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:11.017224 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88862c15-d498-487a-a972-63f905bcbc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:11.017465 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88862c15-d498-487a-a972-63f905bcbc3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:11.018210 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88862c15-d498-487a-a972-63f905bcbc3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:11.019991 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88862c15-d498-487a-a972-63f905bcbc3d admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:11.037966 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1157/2311] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:28:11.039429 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a682bd07-a9e8-4d27-b30b-60279ec262c7 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:11.049598 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a682bd07-a9e8-4d27-b30b-60279ec262c7 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:11.054860 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aed4c4-9daa-47c0-9712-7a908b28186c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:11.055596 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aed4c4-9daa-47c0-9712-7a908b28186c admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:11.055794 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aed4c4-9daa-47c0-9712-7a908b28186c admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:11.056066 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aed4c4-9daa-47c0-9712-7a908b28186c admin admin] PATH_INFO: `/v3/projects/1378813fc9fe4097a92fac06aa97af7a/users/56f7d0db6adc48fa8419e945e4982cb2/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:11.072054 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aed4c4-9daa-47c0-9712-7a908b28186c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1378813fc9fe4097a92fac06aa97af7a, user_id=56f7d0db6adc48fa8419e945e4982cb2, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:11.075121 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aed4c4-9daa-47c0-9712-7a908b28186c admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:11.096135 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1155/2312] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:28:11 2022] PUT /identity/v3/projects/1378813fc9fe4097a92fac06aa97af7a/users/56f7d0db6adc48fa8419e945e4982cb2/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:11.107872 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30ad07-1489-4035-9bb2-96811bdd056a None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:11.108414 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30ad07-1489-4035-9bb2-96811bdd056a None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:11.108527 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30ad07-1489-4035-9bb2-96811bdd056a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:11.147428 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-cd30ad07-1489-4035-9bb2-96811bdd056a None None] MFA Rules not processed for user `56f7d0db6adc48fa8419e945e4982cb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:11.148520 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-cd30ad07-1489-4035-9bb2-96811bdd056a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:11.158837 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aed4c4-9daa-47c0-9712-7a908b28186c admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:11.161224 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-d5aed4c4-9daa-47c0-9712-7a908b28186c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:11.243722 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1158/2313] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:11.252227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30ad07-1489-4035-9bb2-96811bdd056a None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:11.266666 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aed4c4-9daa-47c0-9712-7a908b28186c admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:11.274247 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8683a-9fda-473e-94f4-551c76d6a90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:11.274964 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8683a-9fda-473e-94f4-551c76d6a90c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:11.275247 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8683a-9fda-473e-94f4-551c76d6a90c admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:11.275518 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8683a-9fda-473e-94f4-551c76d6a90c admin admin] PATH_INFO: `/v3/projects/81daffe6f5db4bbf9e66cde38b13f8c0` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:11.276735 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d8683a-9fda-473e-94f4-551c76d6a90c admin admin] RBAC: Authorizing `identity:delete_project(project_id=81daffe6f5db4bbf9e66cde38b13f8c0)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:11.278647 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d8683a-9fda-473e-94f4-551c76d6a90c admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:11.319661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30ad07-1489-4035-9bb2-96811bdd056a None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:11.325953 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-c9d8683a-9fda-473e-94f4-551c76d6a90c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81daffe6f5db4bbf9e66cde38b13f8c0', 'request_id': 'req-c9d8683a-9fda-473e-94f4-551c76d6a90c'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:11.332370 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-c9d8683a-9fda-473e-94f4-551c76d6a90c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81daffe6f5db4bbf9e66cde38b13f8c0', 'request_id': 'req-c9d8683a-9fda-473e-94f4-551c76d6a90c'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:11.338704 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-c9d8683a-9fda-473e-94f4-551c76d6a90c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81daffe6f5db4bbf9e66cde38b13f8c0', 'request_id': 'req-c9d8683a-9fda-473e-94f4-551c76d6a90c'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:11.340369 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:11.350759 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:11.354269 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-c9d8683a-9fda-473e-94f4-551c76d6a90c admin admin] The token cache is being invalidate because project 81daffe6f5db4bbf9e66cde38b13f8c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:28:11.354564 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-c9d8683a-9fda-473e-94f4-551c76d6a90c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:11.355401 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-c9d8683a-9fda-473e-94f4-551c76d6a90c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:11.357668 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1156/2314] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:28:11 2022] DELETE /identity/v3/projects/81daffe6f5db4bbf9e66cde38b13f8c0 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:28:11.362269 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f51f93-4541-4dc5-a358-c3394a67933f None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:11.362582 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f51f93-4541-4dc5-a358-c3394a67933f None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:11.362884 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f51f93-4541-4dc5-a358-c3394a67933f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:11.372419 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d8badb-d245-4587-81d4-28f54d29c7f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:11.373029 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8badb-d245-4587-81d4-28f54d29c7f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:11.373306 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8badb-d245-4587-81d4-28f54d29c7f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:11.373579 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8badb-d245-4587-81d4-28f54d29c7f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:11.375684 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-a0d8badb-d245-4587-81d4-28f54d29c7f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:11.383115 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-95f51f93-4541-4dc5-a358-c3394a67933f None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:11.384309 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-95f51f93-4541-4dc5-a358-c3394a67933f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:11.438456 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d8badb-d245-4587-81d4-28f54d29c7f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:11.440117 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d8badb-d245-4587-81d4-28f54d29c7f8 service cinder] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:11.445049 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1157/2315] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:28:11.451617 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1159/2316] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:28:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 14:28:11.453051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319c48c-75ee-42f5-a7e7-37575453eef2 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:11.453344 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319c48c-75ee-42f5-a7e7-37575453eef2 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:11.453683 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319c48c-75ee-42f5-a7e7-37575453eef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:11.478609 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-f319c48c-75ee-42f5-a7e7-37575453eef2 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:11.478770 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d8badb-d245-4587-81d4-28f54d29c7f8 service cinder] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:11.479964 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-f319c48c-75ee-42f5-a7e7-37575453eef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:11.480079 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-a0d8badb-d245-4587-81d4-28f54d29c7f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:11.491544 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1158/2317] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:28:11.493689 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f319c48c-75ee-42f5-a7e7-37575453eef2 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:11.494582 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d8badb-d245-4587-81d4-28f54d29c7f8 service cinder] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:11.499558 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8e0a18-4998-4529-b25d-60722d4deb0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:11.500193 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e0a18-4998-4529-b25d-60722d4deb0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:11.500525 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e0a18-4998-4529-b25d-60722d4deb0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:11.500797 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e0a18-4998-4529-b25d-60722d4deb0c admin admin] PATH_INFO: `/v3/users/56f7d0db6adc48fa8419e945e4982cb2` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:11.501950 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8e0a18-4998-4529-b25d-60722d4deb0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=56f7d0db6adc48fa8419e945e4982cb2)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:11.503007 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f319c48c-75ee-42f5-a7e7-37575453eef2 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:11.503922 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8e0a18-4998-4529-b25d-60722d4deb0c admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:11.508128 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca14da3-2428-4126-8d10-f6603ae3f7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:11.508714 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca14da3-2428-4126-8d10-f6603ae3f7b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:11.508986 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca14da3-2428-4126-8d10-f6603ae3f7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:11.509253 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca14da3-2428-4126-8d10-f6603ae3f7b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:11.509938 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca14da3-2428-4126-8d10-f6603ae3f7b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:28:11.510236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca14da3-2428-4126-8d10-f6603ae3f7b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:11.511829 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca14da3-2428-4126-8d10-f6603ae3f7b5 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:11.523663 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1159/2318] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:28:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:28:11.527471 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca14da3-2428-4126-8d10-f6603ae3f7b5 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:11.537847 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca14da3-2428-4126-8d10-f6603ae3f7b5 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:11.542805 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21132652-ad0b-4e87-b2ec-f2e539475f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:11.543412 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21132652-ad0b-4e87-b2ec-f2e539475f37 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:11.543689 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21132652-ad0b-4e87-b2ec-f2e539475f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:11.543958 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21132652-ad0b-4e87-b2ec-f2e539475f37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:11.544577 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21132652-ad0b-4e87-b2ec-f2e539475f37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:11.546444 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21132652-ad0b-4e87-b2ec-f2e539475f37 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:11.555649 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-3a8e0a18-4998-4529-b25d-60722d4deb0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56f7d0db6adc48fa8419e945e4982cb2'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:11.556344 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-3a8e0a18-4998-4529-b25d-60722d4deb0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56f7d0db6adc48fa8419e945e4982cb2'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:11.563943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-3a8e0a18-4998-4529-b25d-60722d4deb0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56f7d0db6adc48fa8419e945e4982cb2'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:11.564664 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-3a8e0a18-4998-4529-b25d-60722d4deb0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56f7d0db6adc48fa8419e945e4982cb2'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:11.581147 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-3a8e0a18-4998-4529-b25d-60722d4deb0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56f7d0db6adc48fa8419e945e4982cb2'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:11.585917 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1160/2319] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:28:11 2022] POST /identity/v3/projects => generated 377 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:28:11.590195 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21132652-ad0b-4e87-b2ec-f2e539475f37 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:11.593490 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-21132652-ad0b-4e87-b2ec-f2e539475f37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:11.604670 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1160/2320] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:28:11 2022] DELETE /identity/v3/users/56f7d0db6adc48fa8419e945e4982cb2 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:11.609850 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8f9ba-daac-41c1-8b7b-c6618055866e None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:11.610290 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8f9ba-daac-41c1-8b7b-c6618055866e None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:11.610638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8f9ba-daac-41c1-8b7b-c6618055866e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:11.639219 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-3bb8f9ba-daac-41c1-8b7b-c6618055866e None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:11.641115 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-3bb8f9ba-daac-41c1-8b7b-c6618055866e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:11.680529 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21132652-ad0b-4e87-b2ec-f2e539475f37 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:11.697696 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:11.710950 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1161/2321] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:11.723692 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8f9ba-daac-41c1-8b7b-c6618055866e None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:11.725119 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-3bb8f9ba-daac-41c1-8b7b-c6618055866e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:11.744909 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:INSERT=2,keystone:DELETE=1,keystone:SELECT=97 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:11.756709 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a9110-700c-402e-aed4-ca1a7d5b11a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:11.757424 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a9110-700c-402e-aed4-ca1a7d5b11a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:11.757661 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a9110-700c-402e-aed4-ca1a7d5b11a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:11.757935 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a9110-700c-402e-aed4-ca1a7d5b11a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:11.758598 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5a9110-700c-402e-aed4-ca1a7d5b11a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:11.761053 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5a9110-700c-402e-aed4-ca1a7d5b11a4 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:11.790224 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1161/2322] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:28:11 2022] POST /identity/v3/users => generated 289 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:28:11.791289 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a9110-700c-402e-aed4-ca1a7d5b11a4 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:11.799098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8f9ba-daac-41c1-8b7b-c6618055866e None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:11.801946 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a9110-700c-402e-aed4-ca1a7d5b11a4 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:11.803480 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed173cb3-a5d5-4500-8ccc-8046b4ac4e70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:11.804086 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed173cb3-a5d5-4500-8ccc-8046b4ac4e70 service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:11.804401 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed173cb3-a5d5-4500-8ccc-8046b4ac4e70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:11.805343 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed173cb3-a5d5-4500-8ccc-8046b4ac4e70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:11.808188 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d4b76-2a6e-497f-879d-1162da37169a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:11.808948 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d4b76-2a6e-497f-879d-1162da37169a admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:11.809329 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d4b76-2a6e-497f-879d-1162da37169a admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:11.809769 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d4b76-2a6e-497f-879d-1162da37169a admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:11.810525 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d4b76-2a6e-497f-879d-1162da37169a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:11.813060 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d4b76-2a6e-497f-879d-1162da37169a admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:11.822292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed173cb3-a5d5-4500-8ccc-8046b4ac4e70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:11.822292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed173cb3-a5d5-4500-8ccc-8046b4ac4e70 service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:11.828260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1162/2323] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:28:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 14:28:11.828743 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1162/2324] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:28:11.833539 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed173cb3-a5d5-4500-8ccc-8046b4ac4e70 service neutron] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:11.845120 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed173cb3-a5d5-4500-8ccc-8046b4ac4e70 service neutron] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:11.850465 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e24c80-2323-46e7-9598-0f4b6915c781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:11.851151 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e24c80-2323-46e7-9598-0f4b6915c781 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:11.851423 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e24c80-2323-46e7-9598-0f4b6915c781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:11.851631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e24c80-2323-46e7-9598-0f4b6915c781 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:11.852267 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e24c80-2323-46e7-9598-0f4b6915c781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:11.853991 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e24c80-2323-46e7-9598-0f4b6915c781 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:11.877574 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1163/2325] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:28:11.878947 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d4b76-2a6e-497f-879d-1162da37169a admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:11.891944 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d4b76-2a6e-497f-879d-1162da37169a admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:11.898172 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd6f046-ba55-4ec8-bfe6-ef6eacee8f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:11.898866 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6f046-ba55-4ec8-bfe6-ef6eacee8f6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:11.899143 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6f046-ba55-4ec8-bfe6-ef6eacee8f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:11.899413 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6f046-ba55-4ec8-bfe6-ef6eacee8f6e admin admin] PATH_INFO: `/v3/projects/ce547619e64b46ab8e6bfe15b2237b5d/users/718cb53149764a39a0a3df0bcf21d000/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:11.924728 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd6f046-ba55-4ec8-bfe6-ef6eacee8f6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce547619e64b46ab8e6bfe15b2237b5d, user_id=718cb53149764a39a0a3df0bcf21d000, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:11.927453 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd6f046-ba55-4ec8-bfe6-ef6eacee8f6e admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:11.940412 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1163/2326] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:28:11 2022] PUT /identity/v3/projects/ce547619e64b46ab8e6bfe15b2237b5d/users/718cb53149764a39a0a3df0bcf21d000/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:11.951309 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc598db3-dd7d-4135-aa53-5b3ad03e17dd None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:11.951666 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc598db3-dd7d-4135-aa53-5b3ad03e17dd None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:11.952134 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc598db3-dd7d-4135-aa53-5b3ad03e17dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:11.981938 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-dc598db3-dd7d-4135-aa53-5b3ad03e17dd None None] MFA Rules not processed for user `718cb53149764a39a0a3df0bcf21d000`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:11.983986 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-dc598db3-dd7d-4135-aa53-5b3ad03e17dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:12.052899 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1164/2327] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:28:12.064002 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd6f046-ba55-4ec8-bfe6-ef6eacee8f6e admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:12.065951 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-6dd6f046-ba55-4ec8-bfe6-ef6eacee8f6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:12.129669 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd6f046-ba55-4ec8-bfe6-ef6eacee8f6e admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:12.134040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f75b3-da86-4d36-bd5b-0fcc30f2a3ff service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e82b01dbd6e84a9d82baa2c28d77fc28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:12.134550 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f75b3-da86-4d36-bd5b-0fcc30f2a3ff service glance] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:12.137693 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f75b3-da86-4d36-bd5b-0fcc30f2a3ff service glance] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:12.137693 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f75b3-da86-4d36-bd5b-0fcc30f2a3ff service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:12.138913 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc598db3-dd7d-4135-aa53-5b3ad03e17dd None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:12.140329 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-dc598db3-dd7d-4135-aa53-5b3ad03e17dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:12.152416 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f75b3-da86-4d36-bd5b-0fcc30f2a3ff service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:12.154636 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f75b3-da86-4d36-bd5b-0fcc30f2a3ff service glance] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:12.170337 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1164/2328] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:28:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:28:12.233146 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc598db3-dd7d-4135-aa53-5b3ad03e17dd None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:12.237789 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577caf5b-a946-4f07-a530-18e36d8c9a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:12.238428 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577caf5b-a946-4f07-a530-18e36d8c9a06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:12.238694 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577caf5b-a946-4f07-a530-18e36d8c9a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:12.239012 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577caf5b-a946-4f07-a530-18e36d8c9a06 admin admin] PATH_INFO: `/v3/projects/1378813fc9fe4097a92fac06aa97af7a` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:12.240622 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577caf5b-a946-4f07-a530-18e36d8c9a06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1378813fc9fe4097a92fac06aa97af7a)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:12.242795 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577caf5b-a946-4f07-a530-18e36d8c9a06 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:12.279477 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:12.295657 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-577caf5b-a946-4f07-a530-18e36d8c9a06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1378813fc9fe4097a92fac06aa97af7a', 'request_id': 'req-577caf5b-a946-4f07-a530-18e36d8c9a06'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:12.302227 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-577caf5b-a946-4f07-a530-18e36d8c9a06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1378813fc9fe4097a92fac06aa97af7a', 'request_id': 'req-577caf5b-a946-4f07-a530-18e36d8c9a06'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:12.312311 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-577caf5b-a946-4f07-a530-18e36d8c9a06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1378813fc9fe4097a92fac06aa97af7a', 'request_id': 'req-577caf5b-a946-4f07-a530-18e36d8c9a06'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:12.329677 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-577caf5b-a946-4f07-a530-18e36d8c9a06 admin admin] The token cache is being invalidate because project 1378813fc9fe4097a92fac06aa97af7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:28:12.330284 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-577caf5b-a946-4f07-a530-18e36d8c9a06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:12.331023 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-577caf5b-a946-4f07-a530-18e36d8c9a06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:12.332625 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1165/2329] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:28:12 2022] DELETE /identity/v3/projects/1378813fc9fe4097a92fac06aa97af7a => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:28:22.160369 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=44,keystone:INSERT=4 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:22.325208 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:DELETE=7,keystone:SELECT=15,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:26.556989 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f75b3-da86-4d36-bd5b-0fcc30f2a3ff service glance] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:26.558385 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-483f75b3-da86-4d36-bd5b-0fcc30f2a3ff service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:26.571563 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f75b3-da86-4d36-bd5b-0fcc30f2a3ff service glance] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:26.629351 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e61b461-3d09-4c1a-bd6b-e30e2f3444ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:26.630271 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61b461-3d09-4c1a-bd6b-e30e2f3444ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:26.630271 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61b461-3d09-4c1a-bd6b-e30e2f3444ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:26.630562 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61b461-3d09-4c1a-bd6b-e30e2f3444ca admin admin] PATH_INFO: `/v3/users/08a42da5af7549afab59e966b8da84fa` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:26.631735 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e61b461-3d09-4c1a-bd6b-e30e2f3444ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=08a42da5af7549afab59e966b8da84fa)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:26.633668 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e61b461-3d09-4c1a-bd6b-e30e2f3444ca admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:26.684591 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-7e61b461-3d09-4c1a-bd6b-e30e2f3444ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08a42da5af7549afab59e966b8da84fa'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:26.685312 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-7e61b461-3d09-4c1a-bd6b-e30e2f3444ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08a42da5af7549afab59e966b8da84fa'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:26.685917 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-7e61b461-3d09-4c1a-bd6b-e30e2f3444ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08a42da5af7549afab59e966b8da84fa'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:26.698441 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-7e61b461-3d09-4c1a-bd6b-e30e2f3444ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08a42da5af7549afab59e966b8da84fa'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:26.706251 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-7e61b461-3d09-4c1a-bd6b-e30e2f3444ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08a42da5af7549afab59e966b8da84fa'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:26.721233 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1165/2330] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:28:26 2022] DELETE /identity/v3/users/08a42da5af7549afab59e966b8da84fa => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:26.728338 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5ff3c-fac7-4f6f-b9f8-59c52b93ba98 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:26.728836 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5ff3c-fac7-4f6f-b9f8-59c52b93ba98 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:26.728938 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5ff3c-fac7-4f6f-b9f8-59c52b93ba98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:26.747544 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-a5a5ff3c-fac7-4f6f-b9f8-59c52b93ba98 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:26.748657 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-a5a5ff3c-fac7-4f6f-b9f8-59c52b93ba98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:26.807835 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1166/2331] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:26.818751 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e61b461-3d09-4c1a-bd6b-e30e2f3444ca admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:26.820480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-7e61b461-3d09-4c1a-bd6b-e30e2f3444ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:26.885147 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e61b461-3d09-4c1a-bd6b-e30e2f3444ca admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:26.888956 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b1f93-94c1-4303-aaf6-097a2f61005e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:26.889558 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b1f93-94c1-4303-aaf6-097a2f61005e service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:26.889831 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b1f93-94c1-4303-aaf6-097a2f61005e service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:26.890100 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b1f93-94c1-4303-aaf6-097a2f61005e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:26.900126 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248b1f93-94c1-4303-aaf6-097a2f61005e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:26.901550 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248b1f93-94c1-4303-aaf6-097a2f61005e service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:26.911115 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1166/2332] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:28:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:28:27.211669 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a5ff3c-fac7-4f6f-b9f8-59c52b93ba98 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:27.213232 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-a5a5ff3c-fac7-4f6f-b9f8-59c52b93ba98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:27.228040 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a5ff3c-fac7-4f6f-b9f8-59c52b93ba98 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:27.233215 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8162d9-578b-4523-8ddc-9710c47957b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:27.233845 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8162d9-578b-4523-8ddc-9710c47957b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:27.234158 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8162d9-578b-4523-8ddc-9710c47957b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:27.234425 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8162d9-578b-4523-8ddc-9710c47957b4 admin admin] PATH_INFO: `/v3/projects/f4520940040348c38251ff5a9ac6999f` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:27.236040 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8162d9-578b-4523-8ddc-9710c47957b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4520940040348c38251ff5a9ac6999f)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:27.238041 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8162d9-578b-4523-8ddc-9710c47957b4 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:27.285278 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-ff8162d9-578b-4523-8ddc-9710c47957b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4520940040348c38251ff5a9ac6999f', 'request_id': 'req-ff8162d9-578b-4523-8ddc-9710c47957b4'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:27.292859 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-ff8162d9-578b-4523-8ddc-9710c47957b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4520940040348c38251ff5a9ac6999f', 'request_id': 'req-ff8162d9-578b-4523-8ddc-9710c47957b4'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:27.299759 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-ff8162d9-578b-4523-8ddc-9710c47957b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4520940040348c38251ff5a9ac6999f', 'request_id': 'req-ff8162d9-578b-4523-8ddc-9710c47957b4'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:27.314532 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-ff8162d9-578b-4523-8ddc-9710c47957b4 admin admin] The token cache is being invalidate because project f4520940040348c38251ff5a9ac6999f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:28:27.314945 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-ff8162d9-578b-4523-8ddc-9710c47957b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:27.315643 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-ff8162d9-578b-4523-8ddc-9710c47957b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:27.324115 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1167/2333] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:28:27 2022] DELETE /identity/v3/projects/f4520940040348c38251ff5a9ac6999f => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:34.138493 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b1f93-94c1-4303-aaf6-097a2f61005e service neutron] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:34.139975 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-248b1f93-94c1-4303-aaf6-097a2f61005e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:34.157016 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b1f93-94c1-4303-aaf6-097a2f61005e service neutron] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:34.176820 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:34.218213 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d92f9f6-ca7b-48db-930e-22bff46c2a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:34.218989 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d92f9f6-ca7b-48db-930e-22bff46c2a2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:34.219271 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d92f9f6-ca7b-48db-930e-22bff46c2a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:34.219546 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d92f9f6-ca7b-48db-930e-22bff46c2a2c admin admin] PATH_INFO: `/v3/users/30a350860ec440d0866c9a51f9c5243e` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:34.220654 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d92f9f6-ca7b-48db-930e-22bff46c2a2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=30a350860ec440d0866c9a51f9c5243e)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:34.222564 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d92f9f6-ca7b-48db-930e-22bff46c2a2c admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:34.269831 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9d92f9f6-ca7b-48db-930e-22bff46c2a2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30a350860ec440d0866c9a51f9c5243e'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:34.270309 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9d92f9f6-ca7b-48db-930e-22bff46c2a2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30a350860ec440d0866c9a51f9c5243e'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:34.271051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9d92f9f6-ca7b-48db-930e-22bff46c2a2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30a350860ec440d0866c9a51f9c5243e'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:34.284139 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9d92f9f6-ca7b-48db-930e-22bff46c2a2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30a350860ec440d0866c9a51f9c5243e'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:34.291113 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9d92f9f6-ca7b-48db-930e-22bff46c2a2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30a350860ec440d0866c9a51f9c5243e'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:34.305078 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1167/2334] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:28:34 2022] DELETE /identity/v3/users/30a350860ec440d0866c9a51f9c5243e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:34.309528 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf21d77-cc4d-4f17-8aef-9646d6d61bf4 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:34.310010 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf21d77-cc4d-4f17-8aef-9646d6d61bf4 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:34.310103 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf21d77-cc4d-4f17-8aef-9646d6d61bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:34.330309 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-9bf21d77-cc4d-4f17-8aef-9646d6d61bf4 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:34.331558 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-9bf21d77-cc4d-4f17-8aef-9646d6d61bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:34.391940 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1168/2335] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:34.401823 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d92f9f6-ca7b-48db-930e-22bff46c2a2c admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:34.403301 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-9d92f9f6-ca7b-48db-930e-22bff46c2a2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:34.461766 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d92f9f6-ca7b-48db-930e-22bff46c2a2c admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:34.465430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674b2e4c-cdf6-4bfa-86c9-919bd94f604f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:34.466038 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b2e4c-cdf6-4bfa-86c9-919bd94f604f service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:34.466320 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b2e4c-cdf6-4bfa-86c9-919bd94f604f service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:34.466594 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b2e4c-cdf6-4bfa-86c9-919bd94f604f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:34.476896 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674b2e4c-cdf6-4bfa-86c9-919bd94f604f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:34.478334 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674b2e4c-cdf6-4bfa-86c9-919bd94f604f service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:34.487967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1168/2336] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:28:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:28:34.806967 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf21d77-cc4d-4f17-8aef-9646d6d61bf4 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:34.809538 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-9bf21d77-cc4d-4f17-8aef-9646d6d61bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:34.829453 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf21d77-cc4d-4f17-8aef-9646d6d61bf4 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:34.835087 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096939a0-db95-430c-a445-0a9c42bb97ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:34.835989 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096939a0-db95-430c-a445-0a9c42bb97ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:34.836347 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096939a0-db95-430c-a445-0a9c42bb97ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:34.836679 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096939a0-db95-430c-a445-0a9c42bb97ca admin admin] PATH_INFO: `/v3/projects/6bbceb11e4b346ab86747237c3f27bee` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:34.837884 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096939a0-db95-430c-a445-0a9c42bb97ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bbceb11e4b346ab86747237c3f27bee)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:34.839983 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096939a0-db95-430c-a445-0a9c42bb97ca admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:34.865804 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:34.891339 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-096939a0-db95-430c-a445-0a9c42bb97ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bbceb11e4b346ab86747237c3f27bee', 'request_id': 'req-096939a0-db95-430c-a445-0a9c42bb97ca'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:34.897790 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-096939a0-db95-430c-a445-0a9c42bb97ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bbceb11e4b346ab86747237c3f27bee', 'request_id': 'req-096939a0-db95-430c-a445-0a9c42bb97ca'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:34.903264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-096939a0-db95-430c-a445-0a9c42bb97ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bbceb11e4b346ab86747237c3f27bee', 'request_id': 'req-096939a0-db95-430c-a445-0a9c42bb97ca'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:34.915918 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-096939a0-db95-430c-a445-0a9c42bb97ca admin admin] The token cache is being invalidate because project 6bbceb11e4b346ab86747237c3f27bee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:28:34.916205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-096939a0-db95-430c-a445-0a9c42bb97ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:34.916831 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-096939a0-db95-430c-a445-0a9c42bb97ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:34.918419 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1169/2337] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:28:34 2022] DELETE /identity/v3/projects/6bbceb11e4b346ab86747237c3f27bee => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:34.923441 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb5e4d-5ca7-47e6-b21f-62773bfebf05 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:34.923662 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb5e4d-5ca7-47e6-b21f-62773bfebf05 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:34.923935 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb5e4d-5ca7-47e6-b21f-62773bfebf05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:34.943135 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-12fb5e4d-5ca7-47e6-b21f-62773bfebf05 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:34.944314 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-12fb5e4d-5ca7-47e6-b21f-62773bfebf05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:34.986963 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:35.007365 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1169/2338] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:35.015170 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155e32c-3564-442e-a985-2783cfb6ba7a None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:35.015453 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155e32c-3564-442e-a985-2783cfb6ba7a None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:35.015725 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155e32c-3564-442e-a985-2783cfb6ba7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:35.035779 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-c155e32c-3564-442e-a985-2783cfb6ba7a None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:35.037616 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-c155e32c-3564-442e-a985-2783cfb6ba7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:35.045277 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1170/2339] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:35.049319 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb5e4d-5ca7-47e6-b21f-62773bfebf05 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:35.058793 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb5e4d-5ca7-47e6-b21f-62773bfebf05 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:35.063258 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d44adcf-25a9-465e-a2d4-d2d4f8ab31e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:35.063863 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d44adcf-25a9-465e-a2d4-d2d4f8ab31e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:35.064121 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d44adcf-25a9-465e-a2d4-d2d4f8ab31e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:35.064389 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d44adcf-25a9-465e-a2d4-d2d4f8ab31e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:35.065028 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d44adcf-25a9-465e-a2d4-d2d4f8ab31e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:28:35.065342 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d44adcf-25a9-465e-a2d4-d2d4f8ab31e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:35.066929 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d44adcf-25a9-465e-a2d4-d2d4f8ab31e5 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:35.078273 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1170/2340] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:28:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:28:35.082158 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155e32c-3564-442e-a985-2783cfb6ba7a None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:35.091640 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155e32c-3564-442e-a985-2783cfb6ba7a None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:35.096200 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f9b68d-b53d-4d6f-bc14-28ae300b2e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:35.096774 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f9b68d-b53d-4d6f-bc14-28ae300b2e03 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:35.097047 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f9b68d-b53d-4d6f-bc14-28ae300b2e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:35.097312 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f9b68d-b53d-4d6f-bc14-28ae300b2e03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:35.097925 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f9b68d-b53d-4d6f-bc14-28ae300b2e03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:35.100435 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f9b68d-b53d-4d6f-bc14-28ae300b2e03 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:35.126660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1171/2341] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:28:35 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:28:35.131349 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d44adcf-25a9-465e-a2d4-d2d4f8ab31e5 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:35.141214 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d44adcf-25a9-465e-a2d4-d2d4f8ab31e5 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:35.197036 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68139d7c-0742-4036-8596-3edde73fecd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:35.197561 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68139d7c-0742-4036-8596-3edde73fecd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:35.197838 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68139d7c-0742-4036-8596-3edde73fecd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:35.198107 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68139d7c-0742-4036-8596-3edde73fecd4 admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:35.198735 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68139d7c-0742-4036-8596-3edde73fecd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:35.200714 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68139d7c-0742-4036-8596-3edde73fecd4 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:35.221536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1171/2342] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:28:35 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:28:35.225750 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f9b68d-b53d-4d6f-bc14-28ae300b2e03 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:35.236348 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f9b68d-b53d-4d6f-bc14-28ae300b2e03 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:35.240917 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08771a4-d179-4c4e-9dd4-c37e99c80a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:35.241507 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08771a4-d179-4c4e-9dd4-c37e99c80a2e admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:35.241785 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08771a4-d179-4c4e-9dd4-c37e99c80a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:35.242056 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08771a4-d179-4c4e-9dd4-c37e99c80a2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:35.242689 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08771a4-d179-4c4e-9dd4-c37e99c80a2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:35.244329 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08771a4-d179-4c4e-9dd4-c37e99c80a2e admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:35.254212 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1172/2343] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:28:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:28:35.258501 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68139d7c-0742-4036-8596-3edde73fecd4 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:35.268174 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68139d7c-0742-4036-8596-3edde73fecd4 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:35.273320 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6e38e-c1f8-4df9-8be6-119c85302f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:35.273903 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6e38e-c1f8-4df9-8be6-119c85302f5d admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:35.274181 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6e38e-c1f8-4df9-8be6-119c85302f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:35.274449 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6e38e-c1f8-4df9-8be6-119c85302f5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:35.275122 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab6e38e-c1f8-4df9-8be6-119c85302f5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:35.276696 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab6e38e-c1f8-4df9-8be6-119c85302f5d admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:35.287325 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1172/2344] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:28:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:28:35.291434 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08771a4-d179-4c4e-9dd4-c37e99c80a2e admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:35.301071 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08771a4-d179-4c4e-9dd4-c37e99c80a2e admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:35.305636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061cb179-173c-433b-8dfe-d71a5d1c00b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:35.306265 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061cb179-173c-433b-8dfe-d71a5d1c00b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:35.306544 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061cb179-173c-433b-8dfe-d71a5d1c00b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:35.306838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061cb179-173c-433b-8dfe-d71a5d1c00b4 admin admin] PATH_INFO: `/v3/projects/d8d5f68eccf8427f9f56ed311c39e68a/users/b6ba5928497b43e8a997f71a92500ae3/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:35.322315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061cb179-173c-433b-8dfe-d71a5d1c00b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8d5f68eccf8427f9f56ed311c39e68a, user_id=b6ba5928497b43e8a997f71a92500ae3, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:35.324987 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061cb179-173c-433b-8dfe-d71a5d1c00b4 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:35.338744 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1173/2345] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:28:35 2022] PUT /identity/v3/projects/d8d5f68eccf8427f9f56ed311c39e68a/users/b6ba5928497b43e8a997f71a92500ae3/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:35.349090 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18df47e-ff5c-433f-bffe-56499f92b1c2 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:35.349381 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18df47e-ff5c-433f-bffe-56499f92b1c2 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:35.349675 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18df47e-ff5c-433f-bffe-56499f92b1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:35.376619 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-f18df47e-ff5c-433f-bffe-56499f92b1c2 None None] MFA Rules not processed for user `b6ba5928497b43e8a997f71a92500ae3`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:35.377804 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-f18df47e-ff5c-433f-bffe-56499f92b1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:35.441796 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1173/2346] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:35.446848 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061cb179-173c-433b-8dfe-d71a5d1c00b4 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:35.507747 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061cb179-173c-433b-8dfe-d71a5d1c00b4 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:35.512285 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13658a9b-05e8-4a51-ab8e-7524eccf2eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:35.512879 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13658a9b-05e8-4a51-ab8e-7524eccf2eb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:35.513159 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13658a9b-05e8-4a51-ab8e-7524eccf2eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:35.513432 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13658a9b-05e8-4a51-ab8e-7524eccf2eb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:35.514063 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13658a9b-05e8-4a51-ab8e-7524eccf2eb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:35.516009 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13658a9b-05e8-4a51-ab8e-7524eccf2eb9 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:35.541636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1174/2347] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:28:35 2022] POST /identity/v3/projects => generated 383 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:28:35.545742 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18df47e-ff5c-433f-bffe-56499f92b1c2 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:35.556585 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18df47e-ff5c-433f-bffe-56499f92b1c2 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:35.593154 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:35.612881 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45cc936-cc69-48ca-bf0a-4a46541b282c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:35.613575 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cc936-cc69-48ca-bf0a-4a46541b282c admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:35.614007 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cc936-cc69-48ca-bf0a-4a46541b282c admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:35.614327 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cc936-cc69-48ca-bf0a-4a46541b282c admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:35.616862 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45cc936-cc69-48ca-bf0a-4a46541b282c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:35.617430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45cc936-cc69-48ca-bf0a-4a46541b282c admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:35.639884 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1174/2348] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:28:35 2022] POST /identity/v3/users => generated 298 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:28:35.643708 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13658a9b-05e8-4a51-ab8e-7524eccf2eb9 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:35.653963 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13658a9b-05e8-4a51-ab8e-7524eccf2eb9 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:35.658709 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b9d24-5a60-4779-978f-bd565a503a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:35.659325 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b9d24-5a60-4779-978f-bd565a503a1b admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:35.659600 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b9d24-5a60-4779-978f-bd565a503a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:35.659868 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b9d24-5a60-4779-978f-bd565a503a1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:35.660499 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264b9d24-5a60-4779-978f-bd565a503a1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:35.662091 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264b9d24-5a60-4779-978f-bd565a503a1b admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:35.672284 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1175/2349] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:28:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:28:35.676242 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45cc936-cc69-48ca-bf0a-4a46541b282c admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:35.686916 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45cc936-cc69-48ca-bf0a-4a46541b282c admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:35.691521 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79d44c-ddf3-49db-a585-d46c970a2217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:35.692174 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79d44c-ddf3-49db-a585-d46c970a2217 admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:35.692452 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79d44c-ddf3-49db-a585-d46c970a2217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:35.692725 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79d44c-ddf3-49db-a585-d46c970a2217 admin admin] PATH_INFO: `/v3/projects/9aa53ba228934fcf868f66173c11552a/users/752838287fda4f2ead47b6534af31493/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:35.707834 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee79d44c-ddf3-49db-a585-d46c970a2217 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9aa53ba228934fcf868f66173c11552a, user_id=752838287fda4f2ead47b6534af31493, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:35.710615 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee79d44c-ddf3-49db-a585-d46c970a2217 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:35.722901 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1175/2350] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:28:35 2022] PUT /identity/v3/projects/9aa53ba228934fcf868f66173c11552a/users/752838287fda4f2ead47b6534af31493/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:35.726720 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b9d24-5a60-4779-978f-bd565a503a1b admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:35.736643 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b9d24-5a60-4779-978f-bd565a503a1b admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:35.749467 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:35.803903 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7935be9-a13a-499b-b2bf-5274b4728483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:35.804544 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7935be9-a13a-499b-b2bf-5274b4728483 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:35.804826 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7935be9-a13a-499b-b2bf-5274b4728483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:35.805100 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7935be9-a13a-499b-b2bf-5274b4728483 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:35.805743 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7935be9-a13a-499b-b2bf-5274b4728483 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:35.807378 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7935be9-a13a-499b-b2bf-5274b4728483 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:35.817310 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1176/2351] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:28:35 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:28:35.822037 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79d44c-ddf3-49db-a585-d46c970a2217 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:35.835546 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79d44c-ddf3-49db-a585-d46c970a2217 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:35.840414 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89948f7a-bfb9-4812-90a5-5c7fbdd87e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:35.841190 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89948f7a-bfb9-4812-90a5-5c7fbdd87e9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:35.841605 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89948f7a-bfb9-4812-90a5-5c7fbdd87e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:35.841935 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89948f7a-bfb9-4812-90a5-5c7fbdd87e9d admin admin] PATH_INFO: `/v3/projects/9aa53ba228934fcf868f66173c11552a/users/752838287fda4f2ead47b6534af31493/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:35.844498 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89948f7a-bfb9-4812-90a5-5c7fbdd87e9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9aa53ba228934fcf868f66173c11552a, user_id=752838287fda4f2ead47b6534af31493, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:35.847210 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89948f7a-bfb9-4812-90a5-5c7fbdd87e9d admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:35.858308 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1176/2352] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:28:35 2022] PUT /identity/v3/projects/9aa53ba228934fcf868f66173c11552a/users/752838287fda4f2ead47b6534af31493/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:35.871060 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838ed40-a0d9-4cae-80ce-960d532960dc None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:35.871343 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838ed40-a0d9-4cae-80ce-960d532960dc None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:35.871627 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838ed40-a0d9-4cae-80ce-960d532960dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:35.899356 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-c838ed40-a0d9-4cae-80ce-960d532960dc None None] MFA Rules not processed for user `752838287fda4f2ead47b6534af31493`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:35.900543 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-c838ed40-a0d9-4cae-80ce-960d532960dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:35.968182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1177/2353] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:35 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:35.980773 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89948f7a-bfb9-4812-90a5-5c7fbdd87e9d admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:35.982510 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-89948f7a-bfb9-4812-90a5-5c7fbdd87e9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:36.039525 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89948f7a-bfb9-4812-90a5-5c7fbdd87e9d admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:36.043231 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b5897-434e-421f-ae01-84335f20049a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:36.043827 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b5897-434e-421f-ae01-84335f20049a service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:36.044100 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b5897-434e-421f-ae01-84335f20049a service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:36.044366 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b5897-434e-421f-ae01-84335f20049a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:36.054073 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657b5897-434e-421f-ae01-84335f20049a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:36.055543 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657b5897-434e-421f-ae01-84335f20049a service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:36.064948 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1177/2354] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:28:35 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:28:36.128923 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838ed40-a0d9-4cae-80ce-960d532960dc None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:36.190000 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838ed40-a0d9-4cae-80ce-960d532960dc None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:36.194545 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b05cc0-9bbe-4c9f-8c5a-00651b25d4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:36.195196 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b05cc0-9bbe-4c9f-8c5a-00651b25d4a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:36.195479 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b05cc0-9bbe-4c9f-8c5a-00651b25d4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:36.195750 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b05cc0-9bbe-4c9f-8c5a-00651b25d4a8 admin admin] PATH_INFO: `/v3/users/b6ba5928497b43e8a997f71a92500ae3` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:36.196866 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b05cc0-9bbe-4c9f-8c5a-00651b25d4a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6ba5928497b43e8a997f71a92500ae3)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:36.198783 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b05cc0-9bbe-4c9f-8c5a-00651b25d4a8 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:36.240991 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:36.246728 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-55b05cc0-9bbe-4c9f-8c5a-00651b25d4a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6ba5928497b43e8a997f71a92500ae3'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:36.247476 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-55b05cc0-9bbe-4c9f-8c5a-00651b25d4a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6ba5928497b43e8a997f71a92500ae3'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:36.255151 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-55b05cc0-9bbe-4c9f-8c5a-00651b25d4a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6ba5928497b43e8a997f71a92500ae3'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:36.255787 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-55b05cc0-9bbe-4c9f-8c5a-00651b25d4a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6ba5928497b43e8a997f71a92500ae3'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:36.267615 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-55b05cc0-9bbe-4c9f-8c5a-00651b25d4a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6ba5928497b43e8a997f71a92500ae3'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:36.280310 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1178/2355] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:28:36 2022] DELETE /identity/v3/users/b6ba5928497b43e8a997f71a92500ae3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:36.285037 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e8dba-5e9e-4664-94f2-bc3048e6f370 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:36.285594 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e8dba-5e9e-4664-94f2-bc3048e6f370 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:36.285594 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e8dba-5e9e-4664-94f2-bc3048e6f370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:36.305324 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-397e8dba-5e9e-4664-94f2-bc3048e6f370 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:36.306469 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-397e8dba-5e9e-4664-94f2-bc3048e6f370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:36.369749 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1178/2356] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:36.379434 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b05cc0-9bbe-4c9f-8c5a-00651b25d4a8 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:36.380833 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-55b05cc0-9bbe-4c9f-8c5a-00651b25d4a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:36.438321 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b05cc0-9bbe-4c9f-8c5a-00651b25d4a8 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:36.441862 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17679589-72fb-44e1-804d-91c904cbd18f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:36.442455 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17679589-72fb-44e1-804d-91c904cbd18f service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:36.442730 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17679589-72fb-44e1-804d-91c904cbd18f service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:36.443064 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17679589-72fb-44e1-804d-91c904cbd18f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:36.451897 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17679589-72fb-44e1-804d-91c904cbd18f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:36.453315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17679589-72fb-44e1-804d-91c904cbd18f service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:36.462988 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1179/2357] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:28:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:28:36.724520 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e8dba-5e9e-4664-94f2-bc3048e6f370 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:36.725910 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-397e8dba-5e9e-4664-94f2-bc3048e6f370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:36.738838 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e8dba-5e9e-4664-94f2-bc3048e6f370 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:36.743543 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ecd8bb-07a8-4374-a4b6-00e58ea37ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:36.744178 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecd8bb-07a8-4374-a4b6-00e58ea37ba9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:36.744457 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecd8bb-07a8-4374-a4b6-00e58ea37ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:36.744737 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecd8bb-07a8-4374-a4b6-00e58ea37ba9 admin admin] PATH_INFO: `/v3/projects/d8d5f68eccf8427f9f56ed311c39e68a` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:36.745964 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ecd8bb-07a8-4374-a4b6-00e58ea37ba9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8d5f68eccf8427f9f56ed311c39e68a)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:36.748129 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ecd8bb-07a8-4374-a4b6-00e58ea37ba9 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:36.782209 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:36.797123 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-55ecd8bb-07a8-4374-a4b6-00e58ea37ba9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8d5f68eccf8427f9f56ed311c39e68a', 'request_id': 'req-55ecd8bb-07a8-4374-a4b6-00e58ea37ba9'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:36.803223 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-55ecd8bb-07a8-4374-a4b6-00e58ea37ba9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8d5f68eccf8427f9f56ed311c39e68a', 'request_id': 'req-55ecd8bb-07a8-4374-a4b6-00e58ea37ba9'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:36.809348 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-55ecd8bb-07a8-4374-a4b6-00e58ea37ba9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8d5f68eccf8427f9f56ed311c39e68a', 'request_id': 'req-55ecd8bb-07a8-4374-a4b6-00e58ea37ba9'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:36.826290 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-55ecd8bb-07a8-4374-a4b6-00e58ea37ba9 admin admin] The token cache is being invalidate because project d8d5f68eccf8427f9f56ed311c39e68a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:28:36.826642 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-55ecd8bb-07a8-4374-a4b6-00e58ea37ba9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:36.827521 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-55ecd8bb-07a8-4374-a4b6-00e58ea37ba9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:36.832831 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1179/2358] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:28:36 2022] DELETE /identity/v3/projects/d8d5f68eccf8427f9f56ed311c39e68a => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:36.834846 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17679589-72fb-44e1-804d-91c904cbd18f service neutron] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:36.848397 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17679589-72fb-44e1-804d-91c904cbd18f service neutron] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:36.919517 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f932ff0c-8fa0-4b74-b516-5d682613240d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:36.920152 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f932ff0c-8fa0-4b74-b516-5d682613240d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:36.920434 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f932ff0c-8fa0-4b74-b516-5d682613240d admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:36.920710 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f932ff0c-8fa0-4b74-b516-5d682613240d admin admin] PATH_INFO: `/v3/users/752838287fda4f2ead47b6534af31493` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:36.921850 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f932ff0c-8fa0-4b74-b516-5d682613240d admin admin] RBAC: Authorizing `identity:delete_user(user_id=752838287fda4f2ead47b6534af31493)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:36.923824 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f932ff0c-8fa0-4b74-b516-5d682613240d admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:36.970400 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-f932ff0c-8fa0-4b74-b516-5d682613240d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '752838287fda4f2ead47b6534af31493'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:36.971057 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-f932ff0c-8fa0-4b74-b516-5d682613240d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '752838287fda4f2ead47b6534af31493'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:36.979865 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-f932ff0c-8fa0-4b74-b516-5d682613240d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '752838287fda4f2ead47b6534af31493'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:36.980786 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-f932ff0c-8fa0-4b74-b516-5d682613240d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '752838287fda4f2ead47b6534af31493'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:36.986319 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:36.994536 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-f932ff0c-8fa0-4b74-b516-5d682613240d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '752838287fda4f2ead47b6534af31493'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:37.007533 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1180/2359] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:28:36 2022] DELETE /identity/v3/users/752838287fda4f2ead47b6534af31493 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:37.271822 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ecd8bb-07a8-4374-a4b6-00e58ea37ba9 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:37.273407 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-55ecd8bb-07a8-4374-a4b6-00e58ea37ba9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:37.289705 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ecd8bb-07a8-4374-a4b6-00e58ea37ba9 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:37.346873 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e4d38c-5de9-4bbd-b6f4-c762168d5190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:37.347507 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4d38c-5de9-4bbd-b6f4-c762168d5190 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:37.347785 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4d38c-5de9-4bbd-b6f4-c762168d5190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:37.348057 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4d38c-5de9-4bbd-b6f4-c762168d5190 admin admin] PATH_INFO: `/v3/projects/9aa53ba228934fcf868f66173c11552a` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:37.349158 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e4d38c-5de9-4bbd-b6f4-c762168d5190 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9aa53ba228934fcf868f66173c11552a)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:37.351680 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e4d38c-5de9-4bbd-b6f4-c762168d5190 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:37.396989 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-e7e4d38c-5de9-4bbd-b6f4-c762168d5190 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9aa53ba228934fcf868f66173c11552a', 'request_id': 'req-e7e4d38c-5de9-4bbd-b6f4-c762168d5190'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:37.402898 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-e7e4d38c-5de9-4bbd-b6f4-c762168d5190 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9aa53ba228934fcf868f66173c11552a', 'request_id': 'req-e7e4d38c-5de9-4bbd-b6f4-c762168d5190'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:37.408587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-e7e4d38c-5de9-4bbd-b6f4-c762168d5190 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9aa53ba228934fcf868f66173c11552a', 'request_id': 'req-e7e4d38c-5de9-4bbd-b6f4-c762168d5190'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:37.420523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-e7e4d38c-5de9-4bbd-b6f4-c762168d5190 admin admin] The token cache is being invalidate because project 9aa53ba228934fcf868f66173c11552a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:28:37.420804 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-e7e4d38c-5de9-4bbd-b6f4-c762168d5190 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:37.421417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-e7e4d38c-5de9-4bbd-b6f4-c762168d5190 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:37.422990 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1180/2360] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:28:37 2022] DELETE /identity/v3/projects/9aa53ba228934fcf868f66173c11552a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:37.428445 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd72566-6854-4bec-aa32-207912fa1c4e None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:37.428905 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd72566-6854-4bec-aa32-207912fa1c4e None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:37.429108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd72566-6854-4bec-aa32-207912fa1c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:37.448022 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-5cd72566-6854-4bec-aa32-207912fa1c4e None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:37.449214 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-5cd72566-6854-4bec-aa32-207912fa1c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:37.506453 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1181/2361] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:37.514548 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1e71a-af4f-46b4-b6af-725cf201013c None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:37.514846 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1e71a-af4f-46b4-b6af-725cf201013c None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:37.515126 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1e71a-af4f-46b4-b6af-725cf201013c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:37.534418 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-4fb1e71a-af4f-46b4-b6af-725cf201013c None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:37.535628 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-4fb1e71a-af4f-46b4-b6af-725cf201013c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:37.543874 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1181/2362] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:37.547954 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd72566-6854-4bec-aa32-207912fa1c4e None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:37.558104 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd72566-6854-4bec-aa32-207912fa1c4e None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:37.562656 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d73da05-6539-48e0-b302-bbd6fa8477d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:37.563276 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73da05-6539-48e0-b302-bbd6fa8477d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:37.563555 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73da05-6539-48e0-b302-bbd6fa8477d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:37.563823 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73da05-6539-48e0-b302-bbd6fa8477d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:37.564461 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d73da05-6539-48e0-b302-bbd6fa8477d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:28:37.564758 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d73da05-6539-48e0-b302-bbd6fa8477d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:37.566343 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d73da05-6539-48e0-b302-bbd6fa8477d5 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:37.577580 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1182/2363] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:28:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:28:37.581756 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb1e71a-af4f-46b4-b6af-725cf201013c None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:37.590744 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb1e71a-af4f-46b4-b6af-725cf201013c None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:37.595270 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b7346-27a9-4c15-85cb-39abe246f9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:37.595850 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b7346-27a9-4c15-85cb-39abe246f9d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:37.596124 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b7346-27a9-4c15-85cb-39abe246f9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:37.596390 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b7346-27a9-4c15-85cb-39abe246f9d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:37.597009 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7b7346-27a9-4c15-85cb-39abe246f9d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:37.598890 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7b7346-27a9-4c15-85cb-39abe246f9d5 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:37.626962 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1182/2364] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:28:37 2022] POST /identity/v3/projects => generated 401 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:28:37.631584 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d73da05-6539-48e0-b302-bbd6fa8477d5 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:37.643168 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d73da05-6539-48e0-b302-bbd6fa8477d5 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:37.701720 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b887c6-b7ff-4559-917b-780150d97589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:37.702331 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b887c6-b7ff-4559-917b-780150d97589 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:37.702610 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b887c6-b7ff-4559-917b-780150d97589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:37.702914 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b887c6-b7ff-4559-917b-780150d97589 admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:37.703553 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b887c6-b7ff-4559-917b-780150d97589 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:37.705444 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b887c6-b7ff-4559-917b-780150d97589 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:37.729457 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1183/2365] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:28:37 2022] POST /identity/v3/users => generated 301 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:28:37.730606 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b7346-27a9-4c15-85cb-39abe246f9d5 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:37.740775 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b7346-27a9-4c15-85cb-39abe246f9d5 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:37.745353 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682e3ca4-18ab-4bd0-87ca-a2af599813ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:37.745943 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e3ca4-18ab-4bd0-87ca-a2af599813ba admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:37.746223 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e3ca4-18ab-4bd0-87ca-a2af599813ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:37.746496 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e3ca4-18ab-4bd0-87ca-a2af599813ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:37.747168 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682e3ca4-18ab-4bd0-87ca-a2af599813ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:37.748747 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682e3ca4-18ab-4bd0-87ca-a2af599813ba admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:37.758456 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1183/2366] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:28:37.762300 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b887c6-b7ff-4559-917b-780150d97589 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:37.773126 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b887c6-b7ff-4559-917b-780150d97589 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:37.777697 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0a5ce-44e5-46eb-82a3-3fb7e29a94eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:37.778284 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0a5ce-44e5-46eb-82a3-3fb7e29a94eb admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:37.778563 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0a5ce-44e5-46eb-82a3-3fb7e29a94eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:37.778864 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0a5ce-44e5-46eb-82a3-3fb7e29a94eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:37.779504 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef0a5ce-44e5-46eb-82a3-3fb7e29a94eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:37.781099 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef0a5ce-44e5-46eb-82a3-3fb7e29a94eb admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:37.792510 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1184/2367] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:28:37.796501 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682e3ca4-18ab-4bd0-87ca-a2af599813ba admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:37.806317 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682e3ca4-18ab-4bd0-87ca-a2af599813ba admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:37.811043 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5b43a-57d4-4125-901a-24eaf72c72fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:37.811682 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5b43a-57d4-4125-901a-24eaf72c72fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:37.811961 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5b43a-57d4-4125-901a-24eaf72c72fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:37.812241 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5b43a-57d4-4125-901a-24eaf72c72fb admin admin] PATH_INFO: `/v3/projects/198a9623b4504d7989cac61d9df2d31e/users/c916c88f94ae420290a9d7feae5f69e8/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:37.815763 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:37.829731 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e5b43a-57d4-4125-901a-24eaf72c72fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=198a9623b4504d7989cac61d9df2d31e, user_id=c916c88f94ae420290a9d7feae5f69e8, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:37.832730 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e5b43a-57d4-4125-901a-24eaf72c72fb admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:37.845883 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1184/2368] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:28:37 2022] PUT /identity/v3/projects/198a9623b4504d7989cac61d9df2d31e/users/c916c88f94ae420290a9d7feae5f69e8/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:28:37.858053 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334b544-7432-4953-92c6-1503e780847c None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:37.858366 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334b544-7432-4953-92c6-1503e780847c None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:37.858635 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334b544-7432-4953-92c6-1503e780847c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:37.887323 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-d334b544-7432-4953-92c6-1503e780847c None None] MFA Rules not processed for user `c916c88f94ae420290a9d7feae5f69e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:37.888538 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-d334b544-7432-4953-92c6-1503e780847c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:37.931219 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:37.966752 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1185/2369] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:37 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:37.971344 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5b43a-57d4-4125-901a-24eaf72c72fb admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:38.056122 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5b43a-57d4-4125-901a-24eaf72c72fb admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:38.060716 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f2279f-9154-4342-9e1b-58fe34323b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:38.061314 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2279f-9154-4342-9e1b-58fe34323b2f admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:38.061593 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2279f-9154-4342-9e1b-58fe34323b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:38.061867 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2279f-9154-4342-9e1b-58fe34323b2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:38.062499 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f2279f-9154-4342-9e1b-58fe34323b2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:38.064498 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f2279f-9154-4342-9e1b-58fe34323b2f admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:38.090735 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1185/2370] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:28:37 2022] POST /identity/v3/projects => generated 401 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:28:38.094843 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d334b544-7432-4953-92c6-1503e780847c None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:38.106122 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d334b544-7432-4953-92c6-1503e780847c None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:38.162675 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e565d39-8f03-4319-9507-279e5d6fa149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:38.163312 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e565d39-8f03-4319-9507-279e5d6fa149 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:38.163590 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e565d39-8f03-4319-9507-279e5d6fa149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:38.163857 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e565d39-8f03-4319-9507-279e5d6fa149 admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:38.164489 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e565d39-8f03-4319-9507-279e5d6fa149 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:38.166396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e565d39-8f03-4319-9507-279e5d6fa149 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:38.187225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1186/2371] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:28:38 2022] POST /identity/v3/users => generated 307 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:28:38.191191 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f2279f-9154-4342-9e1b-58fe34323b2f admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:38.201828 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f2279f-9154-4342-9e1b-58fe34323b2f admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:38.206542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34673d5c-a2fa-4a3a-8ebb-1c7275078284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:38.207322 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34673d5c-a2fa-4a3a-8ebb-1c7275078284 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:38.207605 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34673d5c-a2fa-4a3a-8ebb-1c7275078284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:38.207880 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34673d5c-a2fa-4a3a-8ebb-1c7275078284 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:38.208519 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34673d5c-a2fa-4a3a-8ebb-1c7275078284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:38.210915 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34673d5c-a2fa-4a3a-8ebb-1c7275078284 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:38.221666 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1186/2372] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:28:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:28:38.225813 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e565d39-8f03-4319-9507-279e5d6fa149 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:38.235612 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e565d39-8f03-4319-9507-279e5d6fa149 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:38.240093 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3122513-459d-4454-91b1-144c5c9200bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:38.240727 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3122513-459d-4454-91b1-144c5c9200bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:38.241003 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3122513-459d-4454-91b1-144c5c9200bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:38.241271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3122513-459d-4454-91b1-144c5c9200bd admin admin] PATH_INFO: `/v3/projects/0e01f4782f674bd7abe0c966431b1ab1/users/a53ad521c2d94515988dcfe3983641ee/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:38.255996 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3122513-459d-4454-91b1-144c5c9200bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e01f4782f674bd7abe0c966431b1ab1, user_id=a53ad521c2d94515988dcfe3983641ee, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:38.258622 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3122513-459d-4454-91b1-144c5c9200bd admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:38.272490 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1187/2373] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:28:38 2022] PUT /identity/v3/projects/0e01f4782f674bd7abe0c966431b1ab1/users/a53ad521c2d94515988dcfe3983641ee/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:38.276381 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34673d5c-a2fa-4a3a-8ebb-1c7275078284 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:38.286428 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34673d5c-a2fa-4a3a-8ebb-1c7275078284 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:38.345377 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e81b25-451c-4998-9d3f-8522e84b4dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:38.346069 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e81b25-451c-4998-9d3f-8522e84b4dd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:38.346245 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e81b25-451c-4998-9d3f-8522e84b4dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:38.346537 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e81b25-451c-4998-9d3f-8522e84b4dd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:38.347192 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e81b25-451c-4998-9d3f-8522e84b4dd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:38.348772 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e81b25-451c-4998-9d3f-8522e84b4dd6 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:38.359454 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1187/2374] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:28:38 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:28:38.363546 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3122513-459d-4454-91b1-144c5c9200bd admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:38.373716 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3122513-459d-4454-91b1-144c5c9200bd admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:38.378297 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dced4b-bfd4-4a85-a007-1ca5b1bdef39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:38.378994 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dced4b-bfd4-4a85-a007-1ca5b1bdef39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:38.379257 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dced4b-bfd4-4a85-a007-1ca5b1bdef39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:38.379527 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dced4b-bfd4-4a85-a007-1ca5b1bdef39 admin admin] PATH_INFO: `/v3/projects/0e01f4782f674bd7abe0c966431b1ab1/users/a53ad521c2d94515988dcfe3983641ee/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:38.382146 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dced4b-bfd4-4a85-a007-1ca5b1bdef39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e01f4782f674bd7abe0c966431b1ab1, user_id=a53ad521c2d94515988dcfe3983641ee, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:38.384882 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dced4b-bfd4-4a85-a007-1ca5b1bdef39 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:38.394755 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1188/2375] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:28:38 2022] PUT /identity/v3/projects/0e01f4782f674bd7abe0c966431b1ab1/users/a53ad521c2d94515988dcfe3983641ee/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:38.405295 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b3319-8814-4741-9aee-41b5a868fb04 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:38.405590 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b3319-8814-4741-9aee-41b5a868fb04 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:38.405868 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b3319-8814-4741-9aee-41b5a868fb04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:38.433524 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-710b3319-8814-4741-9aee-41b5a868fb04 None None] MFA Rules not processed for user `a53ad521c2d94515988dcfe3983641ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:38.434656 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-710b3319-8814-4741-9aee-41b5a868fb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:38.477543 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:38.502604 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1188/2376] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:38 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:28:38.513850 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dced4b-bfd4-4a85-a007-1ca5b1bdef39 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:38.515545 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-99dced4b-bfd4-4a85-a007-1ca5b1bdef39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:38.574514 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dced4b-bfd4-4a85-a007-1ca5b1bdef39 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:38.578201 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a1ba76-7a00-4445-a4f1-372e9484250f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:38.578794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1ba76-7a00-4445-a4f1-372e9484250f service cinder] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:38.579095 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1ba76-7a00-4445-a4f1-372e9484250f service cinder] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:38.579366 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1ba76-7a00-4445-a4f1-372e9484250f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:38.588273 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a1ba76-7a00-4445-a4f1-372e9484250f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:38.589745 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a1ba76-7a00-4445-a4f1-372e9484250f service cinder] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:38.599241 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1189/2377] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:28:38 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:28:38.677891 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710b3319-8814-4741-9aee-41b5a868fb04 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:38.687796 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710b3319-8814-4741-9aee-41b5a868fb04 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:38.691361 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad6bfbb-1892-4810-b397-f51573f3a1ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:38.691962 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6bfbb-1892-4810-b397-f51573f3a1ab service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:38.692242 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6bfbb-1892-4810-b397-f51573f3a1ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:38.692513 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6bfbb-1892-4810-b397-f51573f3a1ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:38.753940 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad6bfbb-1892-4810-b397-f51573f3a1ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:38.756061 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad6bfbb-1892-4810-b397-f51573f3a1ab service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:38.765063 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1189/2378] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:28:38 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:28:48.594994 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=81,keystone:INSERT=5 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:48.760918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=37 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:51.325909 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a1ba76-7a00-4445-a4f1-372e9484250f service cinder] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:51.404748 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a1ba76-7a00-4445-a4f1-372e9484250f service cinder] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:51.409913 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35757ea-d92d-4d2e-80fc-7665cba3d8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:51.410557 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35757ea-d92d-4d2e-80fc-7665cba3d8d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:51.410882 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35757ea-d92d-4d2e-80fc-7665cba3d8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:51.411175 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35757ea-d92d-4d2e-80fc-7665cba3d8d2 admin admin] PATH_INFO: `/v3/users/c916c88f94ae420290a9d7feae5f69e8` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:51.412296 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35757ea-d92d-4d2e-80fc-7665cba3d8d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c916c88f94ae420290a9d7feae5f69e8)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:51.414825 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35757ea-d92d-4d2e-80fc-7665cba3d8d2 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:51.463531 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-c35757ea-d92d-4d2e-80fc-7665cba3d8d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c916c88f94ae420290a9d7feae5f69e8'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:51.464233 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-c35757ea-d92d-4d2e-80fc-7665cba3d8d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c916c88f94ae420290a9d7feae5f69e8'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:51.471274 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-c35757ea-d92d-4d2e-80fc-7665cba3d8d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c916c88f94ae420290a9d7feae5f69e8'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:51.472018 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-c35757ea-d92d-4d2e-80fc-7665cba3d8d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c916c88f94ae420290a9d7feae5f69e8'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:51.484649 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-c35757ea-d92d-4d2e-80fc-7665cba3d8d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c916c88f94ae420290a9d7feae5f69e8'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:51.497875 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1190/2379] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:28:51 2022] DELETE /identity/v3/users/c916c88f94ae420290a9d7feae5f69e8 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:28:51.502373 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d017d-879b-4e6f-8c75-51ebcb7f104a None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:51.502863 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d017d-879b-4e6f-8c75-51ebcb7f104a None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:51.502971 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d017d-879b-4e6f-8c75-51ebcb7f104a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:51.522747 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-554d017d-879b-4e6f-8c75-51ebcb7f104a None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:51.523973 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-554d017d-879b-4e6f-8c75-51ebcb7f104a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:51.587429 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1190/2380] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:51.594406 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35757ea-d92d-4d2e-80fc-7665cba3d8d2 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:51.595330 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-c35757ea-d92d-4d2e-80fc-7665cba3d8d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:51.643710 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d017d-879b-4e6f-8c75-51ebcb7f104a None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:51.645105 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-554d017d-879b-4e6f-8c75-51ebcb7f104a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:51.654129 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35757ea-d92d-4d2e-80fc-7665cba3d8d2 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:51.658304 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d017d-879b-4e6f-8c75-51ebcb7f104a None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:51.658681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e385c4b-f80f-4d98-a55c-082f71c57157 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:51.659324 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e385c4b-f80f-4d98-a55c-082f71c57157 service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:51.659598 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e385c4b-f80f-4d98-a55c-082f71c57157 service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:51.659868 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e385c4b-f80f-4d98-a55c-082f71c57157 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:51.663638 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677b1d4-74b3-47e3-9eff-654b6317f2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:51.664256 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677b1d4-74b3-47e3-9eff-654b6317f2fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:51.664531 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677b1d4-74b3-47e3-9eff-654b6317f2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:51.664916 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677b1d4-74b3-47e3-9eff-654b6317f2fc admin admin] PATH_INFO: `/v3/users/72bd93f257bf46d29af832f7691dd708` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:51.666035 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d677b1d4-74b3-47e3-9eff-654b6317f2fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=72bd93f257bf46d29af832f7691dd708)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:51.668874 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e385c4b-f80f-4d98-a55c-082f71c57157 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:51.670301 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e385c4b-f80f-4d98-a55c-082f71c57157 service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:51.680108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1191/2381] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:28:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:28:51.681635 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d677b1d4-74b3-47e3-9eff-654b6317f2fc admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:51.765700 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d677b1d4-74b3-47e3-9eff-654b6317f2fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72bd93f257bf46d29af832f7691dd708'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:51.766701 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d677b1d4-74b3-47e3-9eff-654b6317f2fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72bd93f257bf46d29af832f7691dd708'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:51.767545 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d677b1d4-74b3-47e3-9eff-654b6317f2fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72bd93f257bf46d29af832f7691dd708'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:51.784906 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d677b1d4-74b3-47e3-9eff-654b6317f2fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72bd93f257bf46d29af832f7691dd708'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:51.792712 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d677b1d4-74b3-47e3-9eff-654b6317f2fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72bd93f257bf46d29af832f7691dd708'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:51.807843 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1191/2382] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:28:51 2022] DELETE /identity/v3/users/72bd93f257bf46d29af832f7691dd708 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:28:51.812367 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5441ed7-bdac-4fe3-a8de-fdf0ecfc9907 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:51.812675 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5441ed7-bdac-4fe3-a8de-fdf0ecfc9907 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:51.812932 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5441ed7-bdac-4fe3-a8de-fdf0ecfc9907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:51.841034 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-d5441ed7-bdac-4fe3-a8de-fdf0ecfc9907 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:51.842471 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-d5441ed7-bdac-4fe3-a8de-fdf0ecfc9907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:51.861524 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:51.915828 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1192/2383] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:51.926538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677b1d4-74b3-47e3-9eff-654b6317f2fc admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:51.928042 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-d677b1d4-74b3-47e3-9eff-654b6317f2fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:52.001243 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677b1d4-74b3-47e3-9eff-654b6317f2fc admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:52.005811 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5b2aa5-d1c7-42f1-9ed5-37c0605373bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:52.007363 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b2aa5-d1c7-42f1-9ed5-37c0605373bc service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:52.007869 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b2aa5-d1c7-42f1-9ed5-37c0605373bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:52.008811 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b2aa5-d1c7-42f1-9ed5-37c0605373bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:52.013920 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5441ed7-bdac-4fe3-a8de-fdf0ecfc9907 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:52.017678 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-d5441ed7-bdac-4fe3-a8de-fdf0ecfc9907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:52.024036 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5b2aa5-d1c7-42f1-9ed5-37c0605373bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:52.027497 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5b2aa5-d1c7-42f1-9ed5-37c0605373bc service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:52.040039 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1192/2384] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:28:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:28:52.051629 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5441ed7-bdac-4fe3-a8de-fdf0ecfc9907 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:52.057116 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fbae15-d208-4ec1-b2ff-6f4514a8fca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:52.057756 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fbae15-d208-4ec1-b2ff-6f4514a8fca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:52.058034 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fbae15-d208-4ec1-b2ff-6f4514a8fca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:52.058306 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fbae15-d208-4ec1-b2ff-6f4514a8fca9 admin admin] PATH_INFO: `/v3/projects/198a9623b4504d7989cac61d9df2d31e` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:52.059478 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fbae15-d208-4ec1-b2ff-6f4514a8fca9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=198a9623b4504d7989cac61d9df2d31e)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:52.061392 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fbae15-d208-4ec1-b2ff-6f4514a8fca9 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:52.130893 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-f5fbae15-d208-4ec1-b2ff-6f4514a8fca9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '198a9623b4504d7989cac61d9df2d31e', 'request_id': 'req-f5fbae15-d208-4ec1-b2ff-6f4514a8fca9'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:52.138224 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-f5fbae15-d208-4ec1-b2ff-6f4514a8fca9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '198a9623b4504d7989cac61d9df2d31e', 'request_id': 'req-f5fbae15-d208-4ec1-b2ff-6f4514a8fca9'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:52.144569 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-f5fbae15-d208-4ec1-b2ff-6f4514a8fca9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '198a9623b4504d7989cac61d9df2d31e', 'request_id': 'req-f5fbae15-d208-4ec1-b2ff-6f4514a8fca9'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:52.160532 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-f5fbae15-d208-4ec1-b2ff-6f4514a8fca9 admin admin] The token cache is being invalidate because project 198a9623b4504d7989cac61d9df2d31e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:28:52.161019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-f5fbae15-d208-4ec1-b2ff-6f4514a8fca9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:52.161602 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-f5fbae15-d208-4ec1-b2ff-6f4514a8fca9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:52.171404 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1193/2385] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:28:52 2022] DELETE /identity/v3/projects/198a9623b4504d7989cac61d9df2d31e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:52.177610 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5b2aa5-d1c7-42f1-9ed5-37c0605373bc service neutron] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:52.188438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5b2aa5-d1c7-42f1-9ed5-37c0605373bc service neutron] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:52.202799 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:52.261676 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5488011-61b5-465c-983e-e5de516bc9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:52.262427 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5488011-61b5-465c-983e-e5de516bc9ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:52.262773 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5488011-61b5-465c-983e-e5de516bc9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:52.263140 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5488011-61b5-465c-983e-e5de516bc9ff admin admin] PATH_INFO: `/v3/users/a53ad521c2d94515988dcfe3983641ee` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:52.264391 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5488011-61b5-465c-983e-e5de516bc9ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=a53ad521c2d94515988dcfe3983641ee)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:52.266381 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5488011-61b5-465c-983e-e5de516bc9ff admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:52.316094 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-c5488011-61b5-465c-983e-e5de516bc9ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a53ad521c2d94515988dcfe3983641ee'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:52.316771 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-c5488011-61b5-465c-983e-e5de516bc9ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a53ad521c2d94515988dcfe3983641ee'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:52.317433 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-c5488011-61b5-465c-983e-e5de516bc9ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a53ad521c2d94515988dcfe3983641ee'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:52.332133 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-c5488011-61b5-465c-983e-e5de516bc9ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a53ad521c2d94515988dcfe3983641ee'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:52.343229 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fbae15-d208-4ec1-b2ff-6f4514a8fca9 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:52.343864 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-c5488011-61b5-465c-983e-e5de516bc9ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a53ad521c2d94515988dcfe3983641ee'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:52.345645 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-f5fbae15-d208-4ec1-b2ff-6f4514a8fca9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:52.364088 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1193/2386] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:28:52 2022] DELETE /identity/v3/users/a53ad521c2d94515988dcfe3983641ee => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:52.376536 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fbae15-d208-4ec1-b2ff-6f4514a8fca9 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:52.439118 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f1c42-e407-4492-9918-c15bbfa5651f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:52.439764 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f1c42-e407-4492-9918-c15bbfa5651f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:52.440024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f1c42-e407-4492-9918-c15bbfa5651f admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:52.440291 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f1c42-e407-4492-9918-c15bbfa5651f admin admin] PATH_INFO: `/v3/projects/7eaa22c284634760bed017db5a3501c2` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:52.441404 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f1c42-e407-4492-9918-c15bbfa5651f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7eaa22c284634760bed017db5a3501c2)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:52.443330 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f1c42-e407-4492-9918-c15bbfa5651f admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:52.490583 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:52.492878 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-b42f1c42-e407-4492-9918-c15bbfa5651f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7eaa22c284634760bed017db5a3501c2', 'request_id': 'req-b42f1c42-e407-4492-9918-c15bbfa5651f'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:52.501466 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-b42f1c42-e407-4492-9918-c15bbfa5651f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7eaa22c284634760bed017db5a3501c2', 'request_id': 'req-b42f1c42-e407-4492-9918-c15bbfa5651f'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:52.508161 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-b42f1c42-e407-4492-9918-c15bbfa5651f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7eaa22c284634760bed017db5a3501c2', 'request_id': 'req-b42f1c42-e407-4492-9918-c15bbfa5651f'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:52.525444 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-b42f1c42-e407-4492-9918-c15bbfa5651f admin admin] The token cache is being invalidate because project 7eaa22c284634760bed017db5a3501c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:28:52.525743 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-b42f1c42-e407-4492-9918-c15bbfa5651f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:52.526401 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-b42f1c42-e407-4492-9918-c15bbfa5651f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:52.528671 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1194/2387] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:28:52 2022] DELETE /identity/v3/projects/7eaa22c284634760bed017db5a3501c2 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:52.676600 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5488011-61b5-465c-983e-e5de516bc9ff admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:52.678135 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-c5488011-61b5-465c-983e-e5de516bc9ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:52.697976 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5488011-61b5-465c-983e-e5de516bc9ff admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:52.770762 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926d4b4a-cd68-40ad-83b5-7780035c9e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:52.771721 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d4b4a-cd68-40ad-83b5-7780035c9e81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:52.772617 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d4b4a-cd68-40ad-83b5-7780035c9e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:52.772983 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d4b4a-cd68-40ad-83b5-7780035c9e81 admin admin] PATH_INFO: `/v3/projects/0e01f4782f674bd7abe0c966431b1ab1` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:52.774761 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926d4b4a-cd68-40ad-83b5-7780035c9e81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e01f4782f674bd7abe0c966431b1ab1)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:52.776395 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926d4b4a-cd68-40ad-83b5-7780035c9e81 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:52.837089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:52.848236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-926d4b4a-cd68-40ad-83b5-7780035c9e81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e01f4782f674bd7abe0c966431b1ab1', 'request_id': 'req-926d4b4a-cd68-40ad-83b5-7780035c9e81'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:52.855530 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-926d4b4a-cd68-40ad-83b5-7780035c9e81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e01f4782f674bd7abe0c966431b1ab1', 'request_id': 'req-926d4b4a-cd68-40ad-83b5-7780035c9e81'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:52.861593 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-926d4b4a-cd68-40ad-83b5-7780035c9e81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e01f4782f674bd7abe0c966431b1ab1', 'request_id': 'req-926d4b4a-cd68-40ad-83b5-7780035c9e81'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:52.874103 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-926d4b4a-cd68-40ad-83b5-7780035c9e81 admin admin] The token cache is being invalidate because project 0e01f4782f674bd7abe0c966431b1ab1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:28:52.874334 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-926d4b4a-cd68-40ad-83b5-7780035c9e81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:52.875068 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-926d4b4a-cd68-40ad-83b5-7780035c9e81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:52.876698 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1194/2388] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:28:52 2022] DELETE /identity/v3/projects/0e01f4782f674bd7abe0c966431b1ab1 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:52.881943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60107cf6-ce1f-4f6c-814a-87df8ea0841d None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:52.882223 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60107cf6-ce1f-4f6c-814a-87df8ea0841d None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:52.882520 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60107cf6-ce1f-4f6c-814a-87df8ea0841d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:52.903228 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-60107cf6-ce1f-4f6c-814a-87df8ea0841d None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:52.904428 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-60107cf6-ce1f-4f6c-814a-87df8ea0841d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:52.966779 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1195/2389] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:52.974646 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb816d-1016-4ea0-b49b-0b5c715fffbd None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:52.974954 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb816d-1016-4ea0-b49b-0b5c715fffbd None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:52.975782 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb816d-1016-4ea0-b49b-0b5c715fffbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:52.995590 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-7ebb816d-1016-4ea0-b49b-0b5c715fffbd None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:52.996802 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-7ebb816d-1016-4ea0-b49b-0b5c715fffbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:53.004418 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1195/2390] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:53.008480 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60107cf6-ce1f-4f6c-814a-87df8ea0841d None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:53.018448 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60107cf6-ce1f-4f6c-814a-87df8ea0841d None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:53.023621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95a6dc-7525-438b-a649-49cba6cd29cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:53.024210 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95a6dc-7525-438b-a649-49cba6cd29cf admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:53.024519 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95a6dc-7525-438b-a649-49cba6cd29cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:53.024792 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95a6dc-7525-438b-a649-49cba6cd29cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:53.025434 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c95a6dc-7525-438b-a649-49cba6cd29cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:28:53.025738 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c95a6dc-7525-438b-a649-49cba6cd29cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:53.027434 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c95a6dc-7525-438b-a649-49cba6cd29cf admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:53.039934 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1196/2391] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:28:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:28:53.044180 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebb816d-1016-4ea0-b49b-0b5c715fffbd None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:53.053392 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebb816d-1016-4ea0-b49b-0b5c715fffbd None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:53.058377 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d9a52c-f5d3-43dc-b3fb-522f459a6ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:53.058995 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9a52c-f5d3-43dc-b3fb-522f459a6ec8 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:53.059277 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9a52c-f5d3-43dc-b3fb-522f459a6ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:53.059545 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9a52c-f5d3-43dc-b3fb-522f459a6ec8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:53.060164 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d9a52c-f5d3-43dc-b3fb-522f459a6ec8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:53.062036 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d9a52c-f5d3-43dc-b3fb-522f459a6ec8 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:53.089164 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1196/2392] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:28:53 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:28:53.093604 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95a6dc-7525-438b-a649-49cba6cd29cf admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:53.103982 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95a6dc-7525-438b-a649-49cba6cd29cf admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:53.161494 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8e76b-ba0a-4b0a-9c55-2207a40c328f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:53.162094 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8e76b-ba0a-4b0a-9c55-2207a40c328f admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:53.162372 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8e76b-ba0a-4b0a-9c55-2207a40c328f admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:53.162670 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8e76b-ba0a-4b0a-9c55-2207a40c328f admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:53.163326 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb8e76b-ba0a-4b0a-9c55-2207a40c328f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:53.165311 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb8e76b-ba0a-4b0a-9c55-2207a40c328f admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:53.185938 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1197/2393] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:28:53 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:28:53.190154 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d9a52c-f5d3-43dc-b3fb-522f459a6ec8 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:53.201046 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d9a52c-f5d3-43dc-b3fb-522f459a6ec8 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:53.205765 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b346a966-b67a-4fd1-a0c3-728306f58864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:53.206367 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346a966-b67a-4fd1-a0c3-728306f58864 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:53.206662 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346a966-b67a-4fd1-a0c3-728306f58864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:53.206997 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346a966-b67a-4fd1-a0c3-728306f58864 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:53.207635 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b346a966-b67a-4fd1-a0c3-728306f58864 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:53.209199 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b346a966-b67a-4fd1-a0c3-728306f58864 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:53.222494 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1197/2394] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:28:53.223858 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8e76b-ba0a-4b0a-9c55-2207a40c328f admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:53.234186 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8e76b-ba0a-4b0a-9c55-2207a40c328f admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:53.238860 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e233a9-8981-480d-9dbf-da150c2b3ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:53.239476 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e233a9-8981-480d-9dbf-da150c2b3ed5 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:53.239736 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e233a9-8981-480d-9dbf-da150c2b3ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:53.240013 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e233a9-8981-480d-9dbf-da150c2b3ed5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:53.240650 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e233a9-8981-480d-9dbf-da150c2b3ed5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:53.242268 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e233a9-8981-480d-9dbf-da150c2b3ed5 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:53.253804 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1198/2395] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:28:53.258560 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b346a966-b67a-4fd1-a0c3-728306f58864 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:53.268357 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b346a966-b67a-4fd1-a0c3-728306f58864 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:53.273599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df2ac9-3fda-48a3-81b1-9e4e2c800b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:53.274233 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df2ac9-3fda-48a3-81b1-9e4e2c800b52 admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:53.274512 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df2ac9-3fda-48a3-81b1-9e4e2c800b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:53.274782 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df2ac9-3fda-48a3-81b1-9e4e2c800b52 admin admin] PATH_INFO: `/v3/projects/7a4bd1d1e7f64c4d855a0c8a3c976126/users/c348d972d418465a93cb353cce366d25/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:53.289157 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df2ac9-3fda-48a3-81b1-9e4e2c800b52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a4bd1d1e7f64c4d855a0c8a3c976126, user_id=c348d972d418465a93cb353cce366d25, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:53.291849 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df2ac9-3fda-48a3-81b1-9e4e2c800b52 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:53.304805 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1198/2396] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:28:53 2022] PUT /identity/v3/projects/7a4bd1d1e7f64c4d855a0c8a3c976126/users/c348d972d418465a93cb353cce366d25/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:28:53.315927 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26eb88f-abdf-4a20-bba0-951558ee0380 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:53.316211 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26eb88f-abdf-4a20-bba0-951558ee0380 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:53.316481 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26eb88f-abdf-4a20-bba0-951558ee0380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:53.345651 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-f26eb88f-abdf-4a20-bba0-951558ee0380 None None] MFA Rules not processed for user `c348d972d418465a93cb353cce366d25`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:53.346956 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-f26eb88f-abdf-4a20-bba0-951558ee0380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:53.375528 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:53.418309 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1199/2397] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:53 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:53.422560 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df2ac9-3fda-48a3-81b1-9e4e2c800b52 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:53.483918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df2ac9-3fda-48a3-81b1-9e4e2c800b52 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:53.488650 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7270f9e5-ccdd-456e-b4f9-754bf6e7bbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:53.489252 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270f9e5-ccdd-456e-b4f9-754bf6e7bbdf admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:53.489535 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270f9e5-ccdd-456e-b4f9-754bf6e7bbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:53.489810 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270f9e5-ccdd-456e-b4f9-754bf6e7bbdf admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:53.490531 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7270f9e5-ccdd-456e-b4f9-754bf6e7bbdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:53.492962 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7270f9e5-ccdd-456e-b4f9-754bf6e7bbdf admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:53.519008 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1199/2398] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:28:53 2022] POST /identity/v3/projects => generated 387 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:28:53.522993 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26eb88f-abdf-4a20-bba0-951558ee0380 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:53.533902 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26eb88f-abdf-4a20-bba0-951558ee0380 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:53.588752 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61242515-b61c-408f-a1c4-702af1fc367a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:53.589353 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61242515-b61c-408f-a1c4-702af1fc367a admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:53.589635 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61242515-b61c-408f-a1c4-702af1fc367a admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:53.589940 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61242515-b61c-408f-a1c4-702af1fc367a admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:53.590572 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61242515-b61c-408f-a1c4-702af1fc367a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:53.592511 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61242515-b61c-408f-a1c4-702af1fc367a admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:53.613616 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1200/2399] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:28:53 2022] POST /identity/v3/users => generated 300 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:28:53.618211 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7270f9e5-ccdd-456e-b4f9-754bf6e7bbdf admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:53.628985 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7270f9e5-ccdd-456e-b4f9-754bf6e7bbdf admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:53.634156 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e41186-48bb-4bb9-ae89-1de0c6066d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:53.634742 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e41186-48bb-4bb9-ae89-1de0c6066d19 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:53.635061 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e41186-48bb-4bb9-ae89-1de0c6066d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:53.635330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e41186-48bb-4bb9-ae89-1de0c6066d19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:53.635958 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e41186-48bb-4bb9-ae89-1de0c6066d19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:53.637520 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e41186-48bb-4bb9-ae89-1de0c6066d19 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:53.648295 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1200/2400] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:28:53.652487 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61242515-b61c-408f-a1c4-702af1fc367a admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:53.662506 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61242515-b61c-408f-a1c4-702af1fc367a admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:53.667071 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff133a-2c74-469a-b869-a2aca4fdeb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:53.667709 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff133a-2c74-469a-b869-a2aca4fdeb9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:53.667991 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff133a-2c74-469a-b869-a2aca4fdeb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:53.668259 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff133a-2c74-469a-b869-a2aca4fdeb9f admin admin] PATH_INFO: `/v3/projects/b47f1458079c44c48acbc0b4bc8e249e/users/d2b9e2c5ce474692b8867d7b51fa5607/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:53.683083 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cff133a-2c74-469a-b869-a2aca4fdeb9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b47f1458079c44c48acbc0b4bc8e249e, user_id=d2b9e2c5ce474692b8867d7b51fa5607, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:53.685643 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cff133a-2c74-469a-b869-a2aca4fdeb9f admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:53.699926 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1201/2401] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:28:53 2022] PUT /identity/v3/projects/b47f1458079c44c48acbc0b4bc8e249e/users/d2b9e2c5ce474692b8867d7b51fa5607/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:53.703720 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e41186-48bb-4bb9-ae89-1de0c6066d19 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:53.714008 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e41186-48bb-4bb9-ae89-1de0c6066d19 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:53.748829 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:53.778023 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4531af-873a-4c75-b891-804461bd9f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:53.778628 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4531af-873a-4c75-b891-804461bd9f81 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:53.778932 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4531af-873a-4c75-b891-804461bd9f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:53.779204 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4531af-873a-4c75-b891-804461bd9f81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:53.779839 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4531af-873a-4c75-b891-804461bd9f81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:53.781407 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4531af-873a-4c75-b891-804461bd9f81 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:53.791946 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1201/2402] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:28:53.796185 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff133a-2c74-469a-b869-a2aca4fdeb9f admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:53.806440 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff133a-2c74-469a-b869-a2aca4fdeb9f admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:53.811023 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98425004-bd0b-45b7-80c2-1b38d47d379a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:53.811661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98425004-bd0b-45b7-80c2-1b38d47d379a admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:53.811940 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98425004-bd0b-45b7-80c2-1b38d47d379a admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:53.812211 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98425004-bd0b-45b7-80c2-1b38d47d379a admin admin] PATH_INFO: `/v3/projects/b47f1458079c44c48acbc0b4bc8e249e/users/d2b9e2c5ce474692b8867d7b51fa5607/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:53.814185 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98425004-bd0b-45b7-80c2-1b38d47d379a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b47f1458079c44c48acbc0b4bc8e249e, user_id=d2b9e2c5ce474692b8867d7b51fa5607, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:53.816862 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98425004-bd0b-45b7-80c2-1b38d47d379a admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:53.829891 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1202/2403] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:28:53 2022] PUT /identity/v3/projects/b47f1458079c44c48acbc0b4bc8e249e/users/d2b9e2c5ce474692b8867d7b51fa5607/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:53.840779 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a12c6-e19b-46a7-b518-d4f06ae16c58 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:53.841144 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a12c6-e19b-46a7-b518-d4f06ae16c58 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:53.841486 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a12c6-e19b-46a7-b518-d4f06ae16c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:53.876959 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-523a12c6-e19b-46a7-b518-d4f06ae16c58 None None] MFA Rules not processed for user `d2b9e2c5ce474692b8867d7b51fa5607`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:53.878375 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-523a12c6-e19b-46a7-b518-d4f06ae16c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:53.957950 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1202/2404] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:53 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:28:53.969175 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98425004-bd0b-45b7-80c2-1b38d47d379a admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:53.971019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-98425004-bd0b-45b7-80c2-1b38d47d379a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:54.029066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98425004-bd0b-45b7-80c2-1b38d47d379a admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:54.033011 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e01ae5-8cfd-44d3-a70d-8fbef0bb857d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:54.033633 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e01ae5-8cfd-44d3-a70d-8fbef0bb857d service cinder] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:54.034053 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e01ae5-8cfd-44d3-a70d-8fbef0bb857d service cinder] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:54.034329 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e01ae5-8cfd-44d3-a70d-8fbef0bb857d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:54.044278 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e01ae5-8cfd-44d3-a70d-8fbef0bb857d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:54.045698 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e01ae5-8cfd-44d3-a70d-8fbef0bb857d service cinder] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:54.055744 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1203/2405] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:28:53 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:28:54.639517 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523a12c6-e19b-46a7-b518-d4f06ae16c58 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:54.701837 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523a12c6-e19b-46a7-b518-d4f06ae16c58 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:54.712279 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc8e26-2be9-414a-8b59-13c9c8c503db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:54.712929 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc8e26-2be9-414a-8b59-13c9c8c503db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:54.713233 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc8e26-2be9-414a-8b59-13c9c8c503db admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:54.713513 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc8e26-2be9-414a-8b59-13c9c8c503db admin admin] PATH_INFO: `/v3/users/c348d972d418465a93cb353cce366d25` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:54.714693 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbc8e26-2be9-414a-8b59-13c9c8c503db admin admin] RBAC: Authorizing `identity:delete_user(user_id=c348d972d418465a93cb353cce366d25)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:54.716859 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbc8e26-2be9-414a-8b59-13c9c8c503db admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:54.766054 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-dcbc8e26-2be9-414a-8b59-13c9c8c503db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c348d972d418465a93cb353cce366d25'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:54.766692 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-dcbc8e26-2be9-414a-8b59-13c9c8c503db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c348d972d418465a93cb353cce366d25'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:54.767383 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-dcbc8e26-2be9-414a-8b59-13c9c8c503db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c348d972d418465a93cb353cce366d25'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:54.775432 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:54.780589 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-dcbc8e26-2be9-414a-8b59-13c9c8c503db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c348d972d418465a93cb353cce366d25'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:54.788250 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-dcbc8e26-2be9-414a-8b59-13c9c8c503db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c348d972d418465a93cb353cce366d25'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:54.800437 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1203/2406] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:28:54 2022] DELETE /identity/v3/users/c348d972d418465a93cb353cce366d25 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:54.804928 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae518c2f-877d-4d88-98a1-f3a7809c1862 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:54.805422 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae518c2f-877d-4d88-98a1-f3a7809c1862 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:54.805521 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae518c2f-877d-4d88-98a1-f3a7809c1862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:54.824492 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-ae518c2f-877d-4d88-98a1-f3a7809c1862 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:54.825681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-ae518c2f-877d-4d88-98a1-f3a7809c1862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:54.846636 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:54.888181 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1204/2407] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:54.898475 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc8e26-2be9-414a-8b59-13c9c8c503db admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:54.900083 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-dcbc8e26-2be9-414a-8b59-13c9c8c503db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:54.957734 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc8e26-2be9-414a-8b59-13c9c8c503db admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:54.961368 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d2e6c7-95c0-4628-b377-1cb1d0d1ac79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:54.961961 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2e6c7-95c0-4628-b377-1cb1d0d1ac79 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:54.962237 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2e6c7-95c0-4628-b377-1cb1d0d1ac79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:54.962511 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2e6c7-95c0-4628-b377-1cb1d0d1ac79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:54.971344 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d2e6c7-95c0-4628-b377-1cb1d0d1ac79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:54.972693 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d2e6c7-95c0-4628-b377-1cb1d0d1ac79 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:54.982080 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1204/2408] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:28:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:28:55.252608 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae518c2f-877d-4d88-98a1-f3a7809c1862 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:55.254077 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-ae518c2f-877d-4d88-98a1-f3a7809c1862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:55.267730 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae518c2f-877d-4d88-98a1-f3a7809c1862 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:55.273198 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02db8a55-3140-4d1b-ae75-413e12705604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:55.273851 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db8a55-3140-4d1b-ae75-413e12705604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:55.274160 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db8a55-3140-4d1b-ae75-413e12705604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:55.274454 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db8a55-3140-4d1b-ae75-413e12705604 admin admin] PATH_INFO: `/v3/projects/7a4bd1d1e7f64c4d855a0c8a3c976126` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:55.275695 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02db8a55-3140-4d1b-ae75-413e12705604 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a4bd1d1e7f64c4d855a0c8a3c976126)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:55.278768 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02db8a55-3140-4d1b-ae75-413e12705604 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:55.327446 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-02db8a55-3140-4d1b-ae75-413e12705604 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a4bd1d1e7f64c4d855a0c8a3c976126', 'request_id': 'req-02db8a55-3140-4d1b-ae75-413e12705604'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:55.334784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-02db8a55-3140-4d1b-ae75-413e12705604 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a4bd1d1e7f64c4d855a0c8a3c976126', 'request_id': 'req-02db8a55-3140-4d1b-ae75-413e12705604'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:55.341684 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-02db8a55-3140-4d1b-ae75-413e12705604 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a4bd1d1e7f64c4d855a0c8a3c976126', 'request_id': 'req-02db8a55-3140-4d1b-ae75-413e12705604'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:55.356072 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-02db8a55-3140-4d1b-ae75-413e12705604 admin admin] The token cache is being invalidate because project 7a4bd1d1e7f64c4d855a0c8a3c976126 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:28:55.356361 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-02db8a55-3140-4d1b-ae75-413e12705604 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:55.356962 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-02db8a55-3140-4d1b-ae75-413e12705604 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:55.358421 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1205/2409] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:28:55 2022] DELETE /identity/v3/projects/7a4bd1d1e7f64c4d855a0c8a3c976126 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:55.362445 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d2e6c7-95c0-4628-b377-1cb1d0d1ac79 service neutron] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:55.371952 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d2e6c7-95c0-4628-b377-1cb1d0d1ac79 service neutron] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:55.427228 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da79fc-939e-4e40-849c-b84263f510d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:55.427860 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da79fc-939e-4e40-849c-b84263f510d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:55.428150 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da79fc-939e-4e40-849c-b84263f510d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:55.428427 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da79fc-939e-4e40-849c-b84263f510d3 admin admin] PATH_INFO: `/v3/users/d2b9e2c5ce474692b8867d7b51fa5607` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:55.429554 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da79fc-939e-4e40-849c-b84263f510d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2b9e2c5ce474692b8867d7b51fa5607)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:55.431685 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da79fc-939e-4e40-849c-b84263f510d3 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:55.481273 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-c5da79fc-939e-4e40-849c-b84263f510d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2b9e2c5ce474692b8867d7b51fa5607'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:55.481907 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-c5da79fc-939e-4e40-849c-b84263f510d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2b9e2c5ce474692b8867d7b51fa5607'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:55.482507 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-c5da79fc-939e-4e40-849c-b84263f510d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2b9e2c5ce474692b8867d7b51fa5607'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:55.494376 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-c5da79fc-939e-4e40-849c-b84263f510d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2b9e2c5ce474692b8867d7b51fa5607'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:55.501348 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-c5da79fc-939e-4e40-849c-b84263f510d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2b9e2c5ce474692b8867d7b51fa5607'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:55.509540 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:55.515581 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1205/2410] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:28:55 2022] DELETE /identity/v3/users/d2b9e2c5ce474692b8867d7b51fa5607 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:55.776669 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02db8a55-3140-4d1b-ae75-413e12705604 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:55.778188 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-02db8a55-3140-4d1b-ae75-413e12705604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:55.793121 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02db8a55-3140-4d1b-ae75-413e12705604 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:55.852225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29b47e2-7498-4460-9c4f-86c595fc5c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:55.852861 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b47e2-7498-4460-9c4f-86c595fc5c3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:55.853145 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b47e2-7498-4460-9c4f-86c595fc5c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:55.853419 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b47e2-7498-4460-9c4f-86c595fc5c3e admin admin] PATH_INFO: `/v3/projects/b47f1458079c44c48acbc0b4bc8e249e` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:55.854543 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29b47e2-7498-4460-9c4f-86c595fc5c3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b47f1458079c44c48acbc0b4bc8e249e)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:55.856568 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29b47e2-7498-4460-9c4f-86c595fc5c3e admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:55.903432 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-b29b47e2-7498-4460-9c4f-86c595fc5c3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b47f1458079c44c48acbc0b4bc8e249e', 'request_id': 'req-b29b47e2-7498-4460-9c4f-86c595fc5c3e'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:55.905283 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:55.910180 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-b29b47e2-7498-4460-9c4f-86c595fc5c3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b47f1458079c44c48acbc0b4bc8e249e', 'request_id': 'req-b29b47e2-7498-4460-9c4f-86c595fc5c3e'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:55.918537 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-b29b47e2-7498-4460-9c4f-86c595fc5c3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b47f1458079c44c48acbc0b4bc8e249e', 'request_id': 'req-b29b47e2-7498-4460-9c4f-86c595fc5c3e'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:55.931624 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-b29b47e2-7498-4460-9c4f-86c595fc5c3e admin admin] The token cache is being invalidate because project b47f1458079c44c48acbc0b4bc8e249e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:28:55.931935 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-b29b47e2-7498-4460-9c4f-86c595fc5c3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:55.932666 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-b29b47e2-7498-4460-9c4f-86c595fc5c3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:55.934018 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1206/2411] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:28:55 2022] DELETE /identity/v3/projects/b47f1458079c44c48acbc0b4bc8e249e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:55.939222 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a706b-64b0-40d5-a516-7d381ca69695 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:55.939678 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a706b-64b0-40d5-a516-7d381ca69695 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:55.939771 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a706b-64b0-40d5-a516-7d381ca69695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:55.959833 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-1f7a706b-64b0-40d5-a516-7d381ca69695 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:55.961143 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-1f7a706b-64b0-40d5-a516-7d381ca69695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:56.019347 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1206/2412] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:56.027430 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfce359-2a2b-46fb-b1fd-7ab4b2e79a2b None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:56.027715 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfce359-2a2b-46fb-b1fd-7ab4b2e79a2b None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:56.027990 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfce359-2a2b-46fb-b1fd-7ab4b2e79a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:56.050170 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-0dfce359-2a2b-46fb-b1fd-7ab4b2e79a2b None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:56.051423 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-0dfce359-2a2b-46fb-b1fd-7ab4b2e79a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:56.059285 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1207/2413] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:56.063510 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7a706b-64b0-40d5-a516-7d381ca69695 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:56.073446 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7a706b-64b0-40d5-a516-7d381ca69695 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:56.077935 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b22c9-32f7-43b6-a76d-485a928e5a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:56.078524 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b22c9-32f7-43b6-a76d-485a928e5a3a admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:56.078807 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b22c9-32f7-43b6-a76d-485a928e5a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:56.079109 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b22c9-32f7-43b6-a76d-485a928e5a3a admin admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:56.079750 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1b22c9-32f7-43b6-a76d-485a928e5a3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:28:56.080053 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1b22c9-32f7-43b6-a76d-485a928e5a3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:56.081627 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1b22c9-32f7-43b6-a76d-485a928e5a3a admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:56.093494 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1207/2414] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:28:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:28:56.097550 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfce359-2a2b-46fb-b1fd-7ab4b2e79a2b None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:56.106554 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfce359-2a2b-46fb-b1fd-7ab4b2e79a2b None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:56.116390 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7ac6ad-5c3d-4b1c-9fcb-591a08b2336a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:56.117019 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ac6ad-5c3d-4b1c-9fcb-591a08b2336a admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:56.117293 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ac6ad-5c3d-4b1c-9fcb-591a08b2336a admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:56.117584 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ac6ad-5c3d-4b1c-9fcb-591a08b2336a admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:56.118235 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7ac6ad-5c3d-4b1c-9fcb-591a08b2336a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:56.120168 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7ac6ad-5c3d-4b1c-9fcb-591a08b2336a admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:56.146720 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1208/2415] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:28:56 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:28:56.151587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b22c9-32f7-43b6-a76d-485a928e5a3a admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:56.161636 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b22c9-32f7-43b6-a76d-485a928e5a3a admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:56.221410 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e9b6ec-b6f8-44c1-a412-53ff2f71bb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:56.222025 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9b6ec-b6f8-44c1-a412-53ff2f71bb99 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:56.222308 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9b6ec-b6f8-44c1-a412-53ff2f71bb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:56.222586 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9b6ec-b6f8-44c1-a412-53ff2f71bb99 admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:56.223388 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e9b6ec-b6f8-44c1-a412-53ff2f71bb99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:56.225301 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e9b6ec-b6f8-44c1-a412-53ff2f71bb99 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:56.248669 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1208/2416] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:28:56 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:28:56.249797 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7ac6ad-5c3d-4b1c-9fcb-591a08b2336a admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:56.260863 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7ac6ad-5c3d-4b1c-9fcb-591a08b2336a admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:56.265450 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063dd97c-426c-4277-b047-f3a88ce366c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:56.266076 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063dd97c-426c-4277-b047-f3a88ce366c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:56.266358 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063dd97c-426c-4277-b047-f3a88ce366c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:56.266632 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063dd97c-426c-4277-b047-f3a88ce366c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:56.267320 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063dd97c-426c-4277-b047-f3a88ce366c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:56.268915 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063dd97c-426c-4277-b047-f3a88ce366c1 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:56.283376 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1209/2417] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:28:56.284515 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e9b6ec-b6f8-44c1-a412-53ff2f71bb99 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:56.294827 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e9b6ec-b6f8-44c1-a412-53ff2f71bb99 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:56.299500 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f910c-50d3-435d-831d-7842775d262c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:56.300099 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f910c-50d3-435d-831d-7842775d262c admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:56.300374 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f910c-50d3-435d-831d-7842775d262c admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:56.300643 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f910c-50d3-435d-831d-7842775d262c admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:56.301314 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95f910c-50d3-435d-831d-7842775d262c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:56.302934 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95f910c-50d3-435d-831d-7842775d262c admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:56.315727 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1209/2418] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:28:56.320418 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063dd97c-426c-4277-b047-f3a88ce366c1 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:56.330403 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063dd97c-426c-4277-b047-f3a88ce366c1 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:56.335643 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2829a-807c-4aa8-b398-59a8fd127a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:56.336295 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2829a-807c-4aa8-b398-59a8fd127a66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:56.336572 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2829a-807c-4aa8-b398-59a8fd127a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:56.336866 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2829a-807c-4aa8-b398-59a8fd127a66 admin admin] PATH_INFO: `/v3/projects/213211178ab945c2bded60ea046b788f/users/70050725b05f47d886f87285be0092d3/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:56.352684 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a2829a-807c-4aa8-b398-59a8fd127a66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=213211178ab945c2bded60ea046b788f, user_id=70050725b05f47d886f87285be0092d3, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:56.356131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a2829a-807c-4aa8-b398-59a8fd127a66 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:56.374924 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1210/2419] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:28:56 2022] PUT /identity/v3/projects/213211178ab945c2bded60ea046b788f/users/70050725b05f47d886f87285be0092d3/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:56.385886 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f9f07-4d92-422b-808e-ca3d38fa7746 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:56.386176 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f9f07-4d92-422b-808e-ca3d38fa7746 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:56.386452 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f9f07-4d92-422b-808e-ca3d38fa7746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:56.414856 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-289f9f07-4d92-422b-808e-ca3d38fa7746 None None] MFA Rules not processed for user `70050725b05f47d886f87285be0092d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:56.416078 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-289f9f07-4d92-422b-808e-ca3d38fa7746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:56.470422 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:56.482009 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1210/2420] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:56.486515 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2829a-807c-4aa8-b398-59a8fd127a66 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:56.546757 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2829a-807c-4aa8-b398-59a8fd127a66 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:56.551322 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a47d9a-fe52-410c-9809-d1f2119f90d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:56.551914 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a47d9a-fe52-410c-9809-d1f2119f90d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:56.552192 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a47d9a-fe52-410c-9809-d1f2119f90d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:56.552461 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a47d9a-fe52-410c-9809-d1f2119f90d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:56.553080 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a47d9a-fe52-410c-9809-d1f2119f90d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:56.555011 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a47d9a-fe52-410c-9809-d1f2119f90d6 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:56.580287 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1211/2421] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:28:56 2022] POST /identity/v3/projects => generated 381 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:28:56.584975 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f9f07-4d92-422b-808e-ca3d38fa7746 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:56.595401 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f9f07-4d92-422b-808e-ca3d38fa7746 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:56.651667 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2634966-b5eb-4e8e-9d4e-2c4c26e389c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:56.652274 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2634966-b5eb-4e8e-9d4e-2c4c26e389c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:56.652577 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2634966-b5eb-4e8e-9d4e-2c4c26e389c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:56.652849 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2634966-b5eb-4e8e-9d4e-2c4c26e389c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:56.653489 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2634966-b5eb-4e8e-9d4e-2c4c26e389c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:56.655439 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2634966-b5eb-4e8e-9d4e-2c4c26e389c0 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:56.676556 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1211/2422] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:28:56 2022] POST /identity/v3/users => generated 297 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:28:56.680760 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a47d9a-fe52-410c-9809-d1f2119f90d6 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:56.691134 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a47d9a-fe52-410c-9809-d1f2119f90d6 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:56.695644 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f6856-3d6b-4099-8ef9-f18b18f12ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:56.696234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f6856-3d6b-4099-8ef9-f18b18f12ec2 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:56.696508 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f6856-3d6b-4099-8ef9-f18b18f12ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:56.696778 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f6856-3d6b-4099-8ef9-f18b18f12ec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:56.697411 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3f6856-3d6b-4099-8ef9-f18b18f12ec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:56.699027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3f6856-3d6b-4099-8ef9-f18b18f12ec2 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:56.709233 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1212/2423] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:28:56.713640 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2634966-b5eb-4e8e-9d4e-2c4c26e389c0 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:56.723259 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2634966-b5eb-4e8e-9d4e-2c4c26e389c0 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:56.728331 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3eb402-9883-4821-86e4-b8a614ebb93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:56.728967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3eb402-9883-4821-86e4-b8a614ebb93b admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:56.729246 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3eb402-9883-4821-86e4-b8a614ebb93b admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:56.729515 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3eb402-9883-4821-86e4-b8a614ebb93b admin admin] PATH_INFO: `/v3/projects/6e82f55353c54ae3b82162bae9c6455d/users/51bf4c0bf1ac4678b6dd634300db06ea/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:56.744444 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3eb402-9883-4821-86e4-b8a614ebb93b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e82f55353c54ae3b82162bae9c6455d, user_id=51bf4c0bf1ac4678b6dd634300db06ea, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:56.747153 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3eb402-9883-4821-86e4-b8a614ebb93b admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:56.759878 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1212/2424] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:28:56 2022] PUT /identity/v3/projects/6e82f55353c54ae3b82162bae9c6455d/users/51bf4c0bf1ac4678b6dd634300db06ea/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:56.763871 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f6856-3d6b-4099-8ef9-f18b18f12ec2 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:56.774258 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f6856-3d6b-4099-8ef9-f18b18f12ec2 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:56.820736 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:56.832622 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134edaab-7616-4d50-9f29-68bf72868171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:56.833244 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134edaab-7616-4d50-9f29-68bf72868171 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:56.833695 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134edaab-7616-4d50-9f29-68bf72868171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:56.834302 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134edaab-7616-4d50-9f29-68bf72868171 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:56.835782 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134edaab-7616-4d50-9f29-68bf72868171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:56.837544 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134edaab-7616-4d50-9f29-68bf72868171 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:56.847630 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1213/2425] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:28:56.851925 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3eb402-9883-4821-86e4-b8a614ebb93b admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:56.863159 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3eb402-9883-4821-86e4-b8a614ebb93b admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:56.868138 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c799de-d1bc-42a3-9b73-57f1b60366c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:56.868783 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c799de-d1bc-42a3-9b73-57f1b60366c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:56.869065 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c799de-d1bc-42a3-9b73-57f1b60366c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:56.869338 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c799de-d1bc-42a3-9b73-57f1b60366c1 admin admin] PATH_INFO: `/v3/projects/6e82f55353c54ae3b82162bae9c6455d/users/51bf4c0bf1ac4678b6dd634300db06ea/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:56.871327 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c799de-d1bc-42a3-9b73-57f1b60366c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e82f55353c54ae3b82162bae9c6455d, user_id=51bf4c0bf1ac4678b6dd634300db06ea, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:56.873950 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c799de-d1bc-42a3-9b73-57f1b60366c1 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:56.884862 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1213/2426] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:28:56 2022] PUT /identity/v3/projects/6e82f55353c54ae3b82162bae9c6455d/users/51bf4c0bf1ac4678b6dd634300db06ea/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:56.894932 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50bb3f-9cc5-4433-bdca-fda60bb0924a None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:56.895409 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50bb3f-9cc5-4433-bdca-fda60bb0924a None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:56.895501 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50bb3f-9cc5-4433-bdca-fda60bb0924a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:56.923624 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-3a50bb3f-9cc5-4433-bdca-fda60bb0924a None None] MFA Rules not processed for user `51bf4c0bf1ac4678b6dd634300db06ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:56.924804 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-3a50bb3f-9cc5-4433-bdca-fda60bb0924a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:56.993503 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1214/2427] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:56 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:57.005253 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c799de-d1bc-42a3-9b73-57f1b60366c1 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:57.007497 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-83c799de-d1bc-42a3-9b73-57f1b60366c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:57.067401 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c799de-d1bc-42a3-9b73-57f1b60366c1 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:57.072295 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a691518-fe9c-4911-8fb4-7fa5b1a988e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:57.073282 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a691518-fe9c-4911-8fb4-7fa5b1a988e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:57.073805 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a691518-fe9c-4911-8fb4-7fa5b1a988e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:57.074202 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a691518-fe9c-4911-8fb4-7fa5b1a988e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:57.086516 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a691518-fe9c-4911-8fb4-7fa5b1a988e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:57.088392 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a691518-fe9c-4911-8fb4-7fa5b1a988e4 service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:57.099594 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1214/2428] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:28:57 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:28:57.324674 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a50bb3f-9cc5-4433-bdca-fda60bb0924a None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:57.334421 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a50bb3f-9cc5-4433-bdca-fda60bb0924a None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:57.337971 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a4a95-0d32-487d-abf9-0dfd013a452b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:57.338567 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a4a95-0d32-487d-abf9-0dfd013a452b service cinder] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:57.338877 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a4a95-0d32-487d-abf9-0dfd013a452b service cinder] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:57.339156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a4a95-0d32-487d-abf9-0dfd013a452b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:57.396713 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2a4a95-0d32-487d-abf9-0dfd013a452b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:57.398162 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2a4a95-0d32-487d-abf9-0dfd013a452b service cinder] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:57.407625 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1215/2429] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:28:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:28:57.689178 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a691518-fe9c-4911-8fb4-7fa5b1a988e4 service cinder] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:57.738126 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:57.753302 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a691518-fe9c-4911-8fb4-7fa5b1a988e4 service cinder] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:57.758095 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9d415c-f1c6-48ec-9a57-73b9ccdc0c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:57.758710 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d415c-f1c6-48ec-9a57-73b9ccdc0c1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:57.759012 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d415c-f1c6-48ec-9a57-73b9ccdc0c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:57.759284 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d415c-f1c6-48ec-9a57-73b9ccdc0c1c admin admin] PATH_INFO: `/v3/users/70050725b05f47d886f87285be0092d3` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:57.760398 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9d415c-f1c6-48ec-9a57-73b9ccdc0c1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=70050725b05f47d886f87285be0092d3)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:57.762303 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9d415c-f1c6-48ec-9a57-73b9ccdc0c1c admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:57.806125 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-8c9d415c-f1c6-48ec-9a57-73b9ccdc0c1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70050725b05f47d886f87285be0092d3'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:57.806767 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-8c9d415c-f1c6-48ec-9a57-73b9ccdc0c1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70050725b05f47d886f87285be0092d3'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:57.807392 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-8c9d415c-f1c6-48ec-9a57-73b9ccdc0c1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70050725b05f47d886f87285be0092d3'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:57.819268 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-8c9d415c-f1c6-48ec-9a57-73b9ccdc0c1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70050725b05f47d886f87285be0092d3'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:57.826354 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-8c9d415c-f1c6-48ec-9a57-73b9ccdc0c1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70050725b05f47d886f87285be0092d3'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:57.845469 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1215/2430] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:28:57 2022] DELETE /identity/v3/users/70050725b05f47d886f87285be0092d3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:57.850850 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a022d562-a877-4cd3-8672-bb25e122e59f None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:57.851942 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a022d562-a877-4cd3-8672-bb25e122e59f None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:57.851942 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a022d562-a877-4cd3-8672-bb25e122e59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:57.873293 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-a022d562-a877-4cd3-8672-bb25e122e59f None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:57.874426 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-a022d562-a877-4cd3-8672-bb25e122e59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:57.932618 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1216/2431] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:57.942421 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9d415c-f1c6-48ec-9a57-73b9ccdc0c1c admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:57.943885 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-8c9d415c-f1c6-48ec-9a57-73b9ccdc0c1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:58.000675 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9d415c-f1c6-48ec-9a57-73b9ccdc0c1c admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:58.004353 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd1099-e7eb-4298-88d6-0ab1be7e1b8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:58.005062 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd1099-e7eb-4298-88d6-0ab1be7e1b8c service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:58.005340 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd1099-e7eb-4298-88d6-0ab1be7e1b8c service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:58.005874 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd1099-e7eb-4298-88d6-0ab1be7e1b8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:58.014325 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fd1099-e7eb-4298-88d6-0ab1be7e1b8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:58.015785 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fd1099-e7eb-4298-88d6-0ab1be7e1b8c service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:58.025478 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1216/2432] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:28:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:28:58.297659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a022d562-a877-4cd3-8672-bb25e122e59f None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:58.299221 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-a022d562-a877-4cd3-8672-bb25e122e59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:58.313264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a022d562-a877-4cd3-8672-bb25e122e59f None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:58.319217 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63016214-87e6-4825-9ca3-d831832545ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:58.320387 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63016214-87e6-4825-9ca3-d831832545ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:58.320741 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63016214-87e6-4825-9ca3-d831832545ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:58.320921 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63016214-87e6-4825-9ca3-d831832545ad admin admin] PATH_INFO: `/v3/projects/213211178ab945c2bded60ea046b788f` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:58.322046 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63016214-87e6-4825-9ca3-d831832545ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=213211178ab945c2bded60ea046b788f)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:58.324005 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63016214-87e6-4825-9ca3-d831832545ad admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:58.330563 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:58.373648 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-63016214-87e6-4825-9ca3-d831832545ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '213211178ab945c2bded60ea046b788f', 'request_id': 'req-63016214-87e6-4825-9ca3-d831832545ad'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:58.379844 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-63016214-87e6-4825-9ca3-d831832545ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '213211178ab945c2bded60ea046b788f', 'request_id': 'req-63016214-87e6-4825-9ca3-d831832545ad'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:58.385861 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-63016214-87e6-4825-9ca3-d831832545ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '213211178ab945c2bded60ea046b788f', 'request_id': 'req-63016214-87e6-4825-9ca3-d831832545ad'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:58.399383 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-63016214-87e6-4825-9ca3-d831832545ad admin admin] The token cache is being invalidate because project 213211178ab945c2bded60ea046b788f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:28:58.399672 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-63016214-87e6-4825-9ca3-d831832545ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:58.400306 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-63016214-87e6-4825-9ca3-d831832545ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:58.401785 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1217/2433] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:28:58 2022] DELETE /identity/v3/projects/213211178ab945c2bded60ea046b788f => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:58.405742 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd1099-e7eb-4298-88d6-0ab1be7e1b8c service neutron] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:58.415410 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd1099-e7eb-4298-88d6-0ab1be7e1b8c service neutron] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:58.470695 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76acbde4-b2b9-451c-bc08-b70b1e219c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:58.471355 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76acbde4-b2b9-451c-bc08-b70b1e219c9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:58.471635 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76acbde4-b2b9-451c-bc08-b70b1e219c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:58.471907 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76acbde4-b2b9-451c-bc08-b70b1e219c9a admin admin] PATH_INFO: `/v3/users/51bf4c0bf1ac4678b6dd634300db06ea` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:58.473023 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76acbde4-b2b9-451c-bc08-b70b1e219c9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=51bf4c0bf1ac4678b6dd634300db06ea)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:58.474993 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76acbde4-b2b9-451c-bc08-b70b1e219c9a admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:58.486927 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:58.523013 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-76acbde4-b2b9-451c-bc08-b70b1e219c9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51bf4c0bf1ac4678b6dd634300db06ea'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:58.523658 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-76acbde4-b2b9-451c-bc08-b70b1e219c9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51bf4c0bf1ac4678b6dd634300db06ea'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:58.524262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-76acbde4-b2b9-451c-bc08-b70b1e219c9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51bf4c0bf1ac4678b6dd634300db06ea'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:58.536943 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-76acbde4-b2b9-451c-bc08-b70b1e219c9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51bf4c0bf1ac4678b6dd634300db06ea'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:58.543731 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-76acbde4-b2b9-451c-bc08-b70b1e219c9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51bf4c0bf1ac4678b6dd634300db06ea'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:58.556900 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1217/2434] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:28:58 2022] DELETE /identity/v3/users/51bf4c0bf1ac4678b6dd634300db06ea => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:58.891518 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63016214-87e6-4825-9ca3-d831832545ad admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:58.892951 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-63016214-87e6-4825-9ca3-d831832545ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:58.908207 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63016214-87e6-4825-9ca3-d831832545ad admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:58.966345 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23f974-2182-438b-94c6-c9502e2c5e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:58.966999 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23f974-2182-438b-94c6-c9502e2c5e5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:58.967188 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23f974-2182-438b-94c6-c9502e2c5e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:58.967458 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23f974-2182-438b-94c6-c9502e2c5e5a admin admin] PATH_INFO: `/v3/projects/6e82f55353c54ae3b82162bae9c6455d` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:58.968576 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b23f974-2182-438b-94c6-c9502e2c5e5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e82f55353c54ae3b82162bae9c6455d)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:58.970463 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b23f974-2182-438b-94c6-c9502e2c5e5a admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:59.014767 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-6b23f974-2182-438b-94c6-c9502e2c5e5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e82f55353c54ae3b82162bae9c6455d', 'request_id': 'req-6b23f974-2182-438b-94c6-c9502e2c5e5a'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:59.020625 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-6b23f974-2182-438b-94c6-c9502e2c5e5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e82f55353c54ae3b82162bae9c6455d', 'request_id': 'req-6b23f974-2182-438b-94c6-c9502e2c5e5a'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:59.026532 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-6b23f974-2182-438b-94c6-c9502e2c5e5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e82f55353c54ae3b82162bae9c6455d', 'request_id': 'req-6b23f974-2182-438b-94c6-c9502e2c5e5a'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:59.038845 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-6b23f974-2182-438b-94c6-c9502e2c5e5a admin admin] The token cache is being invalidate because project 6e82f55353c54ae3b82162bae9c6455d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:28:59.039135 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-6b23f974-2182-438b-94c6-c9502e2c5e5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:59.039745 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-6b23f974-2182-438b-94c6-c9502e2c5e5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:28:59.041666 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1218/2435] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:28:58 2022] DELETE /identity/v3/projects/6e82f55353c54ae3b82162bae9c6455d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:59.046951 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b919d-ed4c-45b7-86ea-55b2eb262761 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:59.047398 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b919d-ed4c-45b7-86ea-55b2eb262761 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:59.047508 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b919d-ed4c-45b7-86ea-55b2eb262761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:59.066789 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-228b919d-ed4c-45b7-86ea-55b2eb262761 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:59.067994 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-228b919d-ed4c-45b7-86ea-55b2eb262761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:59.128407 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1218/2436] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:59.137444 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e4692-a925-446b-b879-6df3a5278cd9 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:59.137724 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e4692-a925-446b-b879-6df3a5278cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:59.137995 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e4692-a925-446b-b879-6df3a5278cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:59.158300 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-252e4692-a925-446b-b879-6df3a5278cd9 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:59.159591 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-252e4692-a925-446b-b879-6df3a5278cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:59.169111 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1219/2437] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:59.172835 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b919d-ed4c-45b7-86ea-55b2eb262761 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:59.183286 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b919d-ed4c-45b7-86ea-55b2eb262761 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:59.188897 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be69b55-55c8-446b-a6dd-1b02ac5003e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:59.189507 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be69b55-55c8-446b-a6dd-1b02ac5003e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:59.189791 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be69b55-55c8-446b-a6dd-1b02ac5003e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:59.190064 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be69b55-55c8-446b-a6dd-1b02ac5003e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:59.190713 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be69b55-55c8-446b-a6dd-1b02ac5003e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:28:59.191053 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be69b55-55c8-446b-a6dd-1b02ac5003e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:59.192714 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be69b55-55c8-446b-a6dd-1b02ac5003e5 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:59.205401 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1219/2438] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:28:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:28:59.210071 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e4692-a925-446b-b879-6df3a5278cd9 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:59.219475 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e4692-a925-446b-b879-6df3a5278cd9 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:59.224083 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac23fbd-1304-464b-8ab4-1ebff8d22089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:59.224693 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac23fbd-1304-464b-8ab4-1ebff8d22089 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:59.224960 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac23fbd-1304-464b-8ab4-1ebff8d22089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:59.225233 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac23fbd-1304-464b-8ab4-1ebff8d22089 admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:59.225921 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac23fbd-1304-464b-8ab4-1ebff8d22089 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:59.227884 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac23fbd-1304-464b-8ab4-1ebff8d22089 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:59.243644 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:59.255416 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1220/2439] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:28:59 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:28:59.260149 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be69b55-55c8-446b-a6dd-1b02ac5003e5 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:59.270126 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be69b55-55c8-446b-a6dd-1b02ac5003e5 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:59.325207 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16748d2-810d-4581-bc5f-48b67df06093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:59.325795 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16748d2-810d-4581-bc5f-48b67df06093 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:59.326071 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16748d2-810d-4581-bc5f-48b67df06093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:59.326343 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16748d2-810d-4581-bc5f-48b67df06093 admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:59.327005 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16748d2-810d-4581-bc5f-48b67df06093 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:59.328942 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16748d2-810d-4581-bc5f-48b67df06093 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:59.349301 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1220/2440] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:28:59 2022] POST /identity/v3/users => generated 286 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:28:59.353478 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac23fbd-1304-464b-8ab4-1ebff8d22089 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:59.363517 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac23fbd-1304-464b-8ab4-1ebff8d22089 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:59.367974 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91732065-e0ba-45db-bdd3-5a499cdf8002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:59.368560 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91732065-e0ba-45db-bdd3-5a499cdf8002 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:59.368836 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91732065-e0ba-45db-bdd3-5a499cdf8002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:59.369103 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91732065-e0ba-45db-bdd3-5a499cdf8002 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:59.369733 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91732065-e0ba-45db-bdd3-5a499cdf8002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:59.371537 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91732065-e0ba-45db-bdd3-5a499cdf8002 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:59.381951 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1221/2441] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:28:59.385673 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16748d2-810d-4581-bc5f-48b67df06093 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:59.395565 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16748d2-810d-4581-bc5f-48b67df06093 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:59.400752 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6efb0-921f-4e2b-ba49-fc24204d826f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:59.401336 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6efb0-921f-4e2b-ba49-fc24204d826f admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:59.401613 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6efb0-921f-4e2b-ba49-fc24204d826f admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:59.401886 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6efb0-921f-4e2b-ba49-fc24204d826f admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:59.402521 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc6efb0-921f-4e2b-ba49-fc24204d826f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:59.404128 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc6efb0-921f-4e2b-ba49-fc24204d826f admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:59.405324 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:28:59.416924 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1221/2442] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:28:59.420639 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91732065-e0ba-45db-bdd3-5a499cdf8002 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:59.430889 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91732065-e0ba-45db-bdd3-5a499cdf8002 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:59.435392 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4648ce4-1520-4894-9b38-9ff7f6f236a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:59.436062 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4648ce4-1520-4894-9b38-9ff7f6f236a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:59.436349 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4648ce4-1520-4894-9b38-9ff7f6f236a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:59.436823 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4648ce4-1520-4894-9b38-9ff7f6f236a8 admin admin] PATH_INFO: `/v3/projects/cb04a85a9c074d1ba6a1fd2347fc5544/users/1891c1f56ceb4b9f9e90c358966a29a0/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:59.451953 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4648ce4-1520-4894-9b38-9ff7f6f236a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb04a85a9c074d1ba6a1fd2347fc5544, user_id=1891c1f56ceb4b9f9e90c358966a29a0, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:59.455108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4648ce4-1520-4894-9b38-9ff7f6f236a8 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:59.467647 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1222/2443] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:28:59 2022] PUT /identity/v3/projects/cb04a85a9c074d1ba6a1fd2347fc5544/users/1891c1f56ceb4b9f9e90c358966a29a0/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:28:59.472680 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d607774-952c-496e-aa54-8de79cdf519d None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:59.472943 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d607774-952c-496e-aa54-8de79cdf519d None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:59.473218 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d607774-952c-496e-aa54-8de79cdf519d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:59.492176 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-0d607774-952c-496e-aa54-8de79cdf519d None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:28:59.493377 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-0d607774-952c-496e-aa54-8de79cdf519d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:59.551222 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1222/2444] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:28:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:28:59.561311 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4648ce4-1520-4894-9b38-9ff7f6f236a8 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:28:59.563089 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-c4648ce4-1520-4894-9b38-9ff7f6f236a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:28:59.632842 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4648ce4-1520-4894-9b38-9ff7f6f236a8 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:28:59.637317 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cc4221-2873-44cb-ac70-f244a163e159 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:28:59.637912 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc4221-2873-44cb-ac70-f244a163e159 service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:28:59.638212 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc4221-2873-44cb-ac70-f244a163e159 service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:28:59.638481 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc4221-2873-44cb-ac70-f244a163e159 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:28:59.647424 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cc4221-2873-44cb-ac70-f244a163e159 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:28:59.648851 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cc4221-2873-44cb-ac70-f244a163e159 service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:28:59.658271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1223/2445] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:28:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:29:00.329361 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a451d-16db-4e99-92cd-1c2a9243c127 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:00.329890 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a451d-16db-4e99-92cd-1c2a9243c127 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:00.329890 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a451d-16db-4e99-92cd-1c2a9243c127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:00.489067 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-134a451d-16db-4e99-92cd-1c2a9243c127 None None] MFA Rules not processed for user `1891c1f56ceb4b9f9e90c358966a29a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:29:00.490289 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-134a451d-16db-4e99-92cd-1c2a9243c127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:29:00.556427 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1223/2446] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:29:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:29:00.561176 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cc4221-2873-44cb-ac70-f244a163e159 service neutron] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:00.570772 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cc4221-2873-44cb-ac70-f244a163e159 service neutron] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:00.575447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9943fb-403c-478c-9967-d8ecc5b6e989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:00.576042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9943fb-403c-478c-9967-d8ecc5b6e989 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:00.576321 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9943fb-403c-478c-9967-d8ecc5b6e989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:00.576592 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9943fb-403c-478c-9967-d8ecc5b6e989 admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:00.577216 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9943fb-403c-478c-9967-d8ecc5b6e989 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:00.579172 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9943fb-403c-478c-9967-d8ecc5b6e989 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:00.606182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1224/2447] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:29:00 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:29:00.610255 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134a451d-16db-4e99-92cd-1c2a9243c127 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:00.622114 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134a451d-16db-4e99-92cd-1c2a9243c127 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:00.684628 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d204b78-d90a-49cc-a7ea-e2c2c3f78d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:00.685234 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d204b78-d90a-49cc-a7ea-e2c2c3f78d48 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:00.685507 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d204b78-d90a-49cc-a7ea-e2c2c3f78d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:00.685884 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d204b78-d90a-49cc-a7ea-e2c2c3f78d48 admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:00.686539 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d204b78-d90a-49cc-a7ea-e2c2c3f78d48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:00.688449 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d204b78-d90a-49cc-a7ea-e2c2c3f78d48 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:00.709858 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1224/2448] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:29:00 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:29:00.713975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9943fb-403c-478c-9967-d8ecc5b6e989 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:00.724091 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9943fb-403c-478c-9967-d8ecc5b6e989 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:00.729085 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54fd3e-c6cb-4d6b-8502-48a68cec2d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:00.729678 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54fd3e-c6cb-4d6b-8502-48a68cec2d17 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:00.729950 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54fd3e-c6cb-4d6b-8502-48a68cec2d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:00.730214 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54fd3e-c6cb-4d6b-8502-48a68cec2d17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:00.730874 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd54fd3e-c6cb-4d6b-8502-48a68cec2d17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:00.732448 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd54fd3e-c6cb-4d6b-8502-48a68cec2d17 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:00.743458 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1225/2449] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:29:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:29:00.747463 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d204b78-d90a-49cc-a7ea-e2c2c3f78d48 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:00.757716 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d204b78-d90a-49cc-a7ea-e2c2c3f78d48 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:00.762239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144850b-e7b1-4eef-8a27-e60ea4d92af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:00.762915 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144850b-e7b1-4eef-8a27-e60ea4d92af9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:00.763195 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144850b-e7b1-4eef-8a27-e60ea4d92af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:00.763465 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144850b-e7b1-4eef-8a27-e60ea4d92af9 admin admin] PATH_INFO: `/v3/projects/fa1307e6cbc2454bb1eea745990a3a53/users/9e3028d8dd8b4554a8ff4e1f9355a054/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:00.775164 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:29:00.778459 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e144850b-e7b1-4eef-8a27-e60ea4d92af9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa1307e6cbc2454bb1eea745990a3a53, user_id=9e3028d8dd8b4554a8ff4e1f9355a054, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:00.781223 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e144850b-e7b1-4eef-8a27-e60ea4d92af9 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:00.796623 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1225/2450] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:29:00 2022] PUT /identity/v3/projects/fa1307e6cbc2454bb1eea745990a3a53/users/9e3028d8dd8b4554a8ff4e1f9355a054/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:29:00.800542 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54fd3e-c6cb-4d6b-8502-48a68cec2d17 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:00.825126 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54fd3e-c6cb-4d6b-8502-48a68cec2d17 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:00.899229 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc8002c-d7e6-4252-b6ef-55fa27ce4260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:00.899846 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc8002c-d7e6-4252-b6ef-55fa27ce4260 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:00.900129 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc8002c-d7e6-4252-b6ef-55fa27ce4260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:00.900400 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc8002c-d7e6-4252-b6ef-55fa27ce4260 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:00.901045 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc8002c-d7e6-4252-b6ef-55fa27ce4260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:00.902640 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc8002c-d7e6-4252-b6ef-55fa27ce4260 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:00.913813 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1226/2451] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:29:00 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:29:00.917941 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144850b-e7b1-4eef-8a27-e60ea4d92af9 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:00.928942 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144850b-e7b1-4eef-8a27-e60ea4d92af9 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:00.933635 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511867d6-8ef9-437d-be0f-c2771cafe33b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:00.934276 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511867d6-8ef9-437d-be0f-c2771cafe33b admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:00.934552 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511867d6-8ef9-437d-be0f-c2771cafe33b admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:00.934840 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511867d6-8ef9-437d-be0f-c2771cafe33b admin admin] PATH_INFO: `/v3/projects/fa1307e6cbc2454bb1eea745990a3a53/users/9e3028d8dd8b4554a8ff4e1f9355a054/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:00.936723 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511867d6-8ef9-437d-be0f-c2771cafe33b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa1307e6cbc2454bb1eea745990a3a53, user_id=9e3028d8dd8b4554a8ff4e1f9355a054, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:00.939371 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511867d6-8ef9-437d-be0f-c2771cafe33b admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:00.949721 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1226/2452] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:29:00 2022] PUT /identity/v3/projects/fa1307e6cbc2454bb1eea745990a3a53/users/9e3028d8dd8b4554a8ff4e1f9355a054/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:29:01.662317 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1373a-67bc-4379-9850-615d13705fac None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:01.662847 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1373a-67bc-4379-9850-615d13705fac None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:01.662942 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1373a-67bc-4379-9850-615d13705fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:01.699156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-afe1373a-67bc-4379-9850-615d13705fac None None] MFA Rules not processed for user `9e3028d8dd8b4554a8ff4e1f9355a054`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:29:01.700436 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-afe1373a-67bc-4379-9850-615d13705fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:29:01.715175 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:29:01.787224 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1227/2453] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:29:01 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:29:01.802440 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511867d6-8ef9-437d-be0f-c2771cafe33b admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:01.806507 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-511867d6-8ef9-437d-be0f-c2771cafe33b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:29:01.877428 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511867d6-8ef9-437d-be0f-c2771cafe33b admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:01.882122 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f65cdfa-05e8-43de-b319-81c72741a2e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:01.882731 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65cdfa-05e8-43de-b319-81c72741a2e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:01.883042 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65cdfa-05e8-43de-b319-81c72741a2e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:01.883305 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65cdfa-05e8-43de-b319-81c72741a2e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:01.942990 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f65cdfa-05e8-43de-b319-81c72741a2e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:01.945147 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f65cdfa-05e8-43de-b319-81c72741a2e6 service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:01.954608 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1227/2454] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:29:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:29:08.620179 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1373a-67bc-4379-9850-615d13705fac None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:08.630357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1373a-67bc-4379-9850-615d13705fac None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:08.634415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b76cf-ae47-4018-b278-d478039b1b6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:08.635053 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b76cf-ae47-4018-b278-d478039b1b6a service cinder] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:08.635327 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b76cf-ae47-4018-b278-d478039b1b6a service cinder] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:08.635617 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b76cf-ae47-4018-b278-d478039b1b6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:08.644760 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86b76cf-ae47-4018-b278-d478039b1b6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:08.646185 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86b76cf-ae47-4018-b278-d478039b1b6a service cinder] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:08.655791 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1228/2455] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:29:08 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:29:08.957765 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f65cdfa-05e8-43de-b319-81c72741a2e6 service cinder] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:08.959354 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-2f65cdfa-05e8-43de-b319-81c72741a2e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:29:09.022725 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f65cdfa-05e8-43de-b319-81c72741a2e6 service cinder] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:09.027284 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac6ebc-e938-4283-b66b-af355a2817a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:09.027910 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac6ebc-e938-4283-b66b-af355a2817a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:09.028186 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac6ebc-e938-4283-b66b-af355a2817a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:09.028457 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac6ebc-e938-4283-b66b-af355a2817a2 admin admin] PATH_INFO: `/v3/users/718cb53149764a39a0a3df0bcf21d000` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:09.029570 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bac6ebc-e938-4283-b66b-af355a2817a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=718cb53149764a39a0a3df0bcf21d000)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:09.031517 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bac6ebc-e938-4283-b66b-af355a2817a2 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:09.067630 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:29:09.077709 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-7bac6ebc-e938-4283-b66b-af355a2817a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '718cb53149764a39a0a3df0bcf21d000'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:29:09.078316 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-7bac6ebc-e938-4283-b66b-af355a2817a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '718cb53149764a39a0a3df0bcf21d000'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:29:09.078960 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-7bac6ebc-e938-4283-b66b-af355a2817a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '718cb53149764a39a0a3df0bcf21d000'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:29:09.090973 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-7bac6ebc-e938-4283-b66b-af355a2817a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '718cb53149764a39a0a3df0bcf21d000'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:29:09.097766 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-7bac6ebc-e938-4283-b66b-af355a2817a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '718cb53149764a39a0a3df0bcf21d000'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:29:09.110897 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1228/2456] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:29:08 2022] DELETE /identity/v3/users/718cb53149764a39a0a3df0bcf21d000 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:29:09.116121 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11ff7a-8a1e-4e4c-8cfe-651728a0c067 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:09.116410 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11ff7a-8a1e-4e4c-8cfe-651728a0c067 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:09.116689 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11ff7a-8a1e-4e4c-8cfe-651728a0c067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:09.135510 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-8f11ff7a-8a1e-4e4c-8cfe-651728a0c067 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:29:09.136705 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-8f11ff7a-8a1e-4e4c-8cfe-651728a0c067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:29:09.206875 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1229/2457] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:29:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:29:09.217067 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac6ebc-e938-4283-b66b-af355a2817a2 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:09.218482 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-7bac6ebc-e938-4283-b66b-af355a2817a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:29:09.275184 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac6ebc-e938-4283-b66b-af355a2817a2 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:09.278754 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355006aa-19b3-4bb2-a63f-16a3540fc549 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:09.279382 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355006aa-19b3-4bb2-a63f-16a3540fc549 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:09.279686 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355006aa-19b3-4bb2-a63f-16a3540fc549 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:09.279960 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355006aa-19b3-4bb2-a63f-16a3540fc549 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:09.289729 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355006aa-19b3-4bb2-a63f-16a3540fc549 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:09.291214 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355006aa-19b3-4bb2-a63f-16a3540fc549 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:09.300344 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1229/2458] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:29:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:29:09.576582 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f11ff7a-8a1e-4e4c-8cfe-651728a0c067 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:09.578016 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-8f11ff7a-8a1e-4e4c-8cfe-651728a0c067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:29:09.591271 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f11ff7a-8a1e-4e4c-8cfe-651728a0c067 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:09.596957 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf222464-17df-4d1f-9172-7fc50e0c3e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:09.597592 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf222464-17df-4d1f-9172-7fc50e0c3e45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:09.597867 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf222464-17df-4d1f-9172-7fc50e0c3e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:09.598293 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf222464-17df-4d1f-9172-7fc50e0c3e45 admin admin] PATH_INFO: `/v3/projects/ce547619e64b46ab8e6bfe15b2237b5d` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:09.599438 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf222464-17df-4d1f-9172-7fc50e0c3e45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce547619e64b46ab8e6bfe15b2237b5d)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:09.601351 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf222464-17df-4d1f-9172-7fc50e0c3e45 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:09.656177 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-bf222464-17df-4d1f-9172-7fc50e0c3e45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce547619e64b46ab8e6bfe15b2237b5d', 'request_id': 'req-bf222464-17df-4d1f-9172-7fc50e0c3e45'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:29:09.662537 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-bf222464-17df-4d1f-9172-7fc50e0c3e45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce547619e64b46ab8e6bfe15b2237b5d', 'request_id': 'req-bf222464-17df-4d1f-9172-7fc50e0c3e45'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:29:09.668177 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-bf222464-17df-4d1f-9172-7fc50e0c3e45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce547619e64b46ab8e6bfe15b2237b5d', 'request_id': 'req-bf222464-17df-4d1f-9172-7fc50e0c3e45'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:29:09.680787 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-bf222464-17df-4d1f-9172-7fc50e0c3e45 admin admin] The token cache is being invalidate because project ce547619e64b46ab8e6bfe15b2237b5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:29:09.681110 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-bf222464-17df-4d1f-9172-7fc50e0c3e45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:29:09.681728 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-bf222464-17df-4d1f-9172-7fc50e0c3e45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:29:09.683332 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1230/2459] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:29:09 2022] DELETE /identity/v3/projects/ce547619e64b46ab8e6bfe15b2237b5d => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:29:09.688297 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31f393-5742-4a40-af95-3d01339b0d00 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:09.688581 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31f393-5742-4a40-af95-3d01339b0d00 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:09.688850 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31f393-5742-4a40-af95-3d01339b0d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:09.708263 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-4c31f393-5742-4a40-af95-3d01339b0d00 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:29:09.710018 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-4c31f393-5742-4a40-af95-3d01339b0d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:29:09.769245 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1230/2460] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:29:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:29:09.777800 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd1854-2cb9-406d-b670-4a8b3541d485 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:09.778119 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd1854-2cb9-406d-b670-4a8b3541d485 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:09.778357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd1854-2cb9-406d-b670-4a8b3541d485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:09.799942 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-cbcd1854-2cb9-406d-b670-4a8b3541d485 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:29:09.801184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-cbcd1854-2cb9-406d-b670-4a8b3541d485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:29:09.809550 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1231/2461] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:29:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:29:09.813810 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c31f393-5742-4a40-af95-3d01339b0d00 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:09.824537 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c31f393-5742-4a40-af95-3d01339b0d00 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:09.829458 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26263be9-d4ee-45e7-9224-40ac60988af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:09.830113 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26263be9-d4ee-45e7-9224-40ac60988af5 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:09.830379 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26263be9-d4ee-45e7-9224-40ac60988af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:09.830646 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26263be9-d4ee-45e7-9224-40ac60988af5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:09.831551 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26263be9-d4ee-45e7-9224-40ac60988af5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:29:09.831853 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26263be9-d4ee-45e7-9224-40ac60988af5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:09.833480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26263be9-d4ee-45e7-9224-40ac60988af5 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:09.847464 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1231/2462] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:29:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:29:09.853104 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcd1854-2cb9-406d-b670-4a8b3541d485 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:09.863215 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcd1854-2cb9-406d-b670-4a8b3541d485 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:09.868249 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9340d01a-8229-46ae-924e-5f07d57f0055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:09.868804 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340d01a-8229-46ae-924e-5f07d57f0055 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:09.869086 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340d01a-8229-46ae-924e-5f07d57f0055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:09.869350 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340d01a-8229-46ae-924e-5f07d57f0055 admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:09.870006 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9340d01a-8229-46ae-924e-5f07d57f0055 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:09.871994 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9340d01a-8229-46ae-924e-5f07d57f0055 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:09.890610 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:29:09.912359 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1232/2463] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:29:09 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:29:09.913540 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26263be9-d4ee-45e7-9224-40ac60988af5 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:09.925528 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26263be9-d4ee-45e7-9224-40ac60988af5 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:09.973275 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:29:09.992021 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30c777-7186-4fda-af93-8d32a4849b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:09.992638 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30c777-7186-4fda-af93-8d32a4849b2f admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:09.992915 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30c777-7186-4fda-af93-8d32a4849b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:09.993185 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30c777-7186-4fda-af93-8d32a4849b2f admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:09.993812 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30c777-7186-4fda-af93-8d32a4849b2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:09.995775 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30c777-7186-4fda-af93-8d32a4849b2f admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:10.017822 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1232/2464] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:29:09 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:29:10.021890 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9340d01a-8229-46ae-924e-5f07d57f0055 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:10.033070 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9340d01a-8229-46ae-924e-5f07d57f0055 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:10.037875 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec1452-1370-4aab-8b8d-59eb5ae5cc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:10.038486 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec1452-1370-4aab-8b8d-59eb5ae5cc38 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:10.038762 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec1452-1370-4aab-8b8d-59eb5ae5cc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:10.046174 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec1452-1370-4aab-8b8d-59eb5ae5cc38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:10.046174 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec1452-1370-4aab-8b8d-59eb5ae5cc38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:10.046174 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec1452-1370-4aab-8b8d-59eb5ae5cc38 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:10.051824 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1233/2465] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:29:10.056351 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30c777-7186-4fda-af93-8d32a4849b2f admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:10.066728 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30c777-7186-4fda-af93-8d32a4849b2f admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:10.071937 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c766da01-7f49-49e1-99ff-58d597b55ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:10.072531 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c766da01-7f49-49e1-99ff-58d597b55ab4 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:10.072807 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c766da01-7f49-49e1-99ff-58d597b55ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:10.073077 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c766da01-7f49-49e1-99ff-58d597b55ab4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:10.073715 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c766da01-7f49-49e1-99ff-58d597b55ab4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:10.075339 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c766da01-7f49-49e1-99ff-58d597b55ab4 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:10.085634 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1233/2466] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 14:29:10.089738 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec1452-1370-4aab-8b8d-59eb5ae5cc38 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:10.099633 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec1452-1370-4aab-8b8d-59eb5ae5cc38 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:10.104187 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4266b-260f-43ec-9d6f-b49fb7239dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:10.104823 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4266b-260f-43ec-9d6f-b49fb7239dda admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:10.105102 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4266b-260f-43ec-9d6f-b49fb7239dda admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:10.105369 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4266b-260f-43ec-9d6f-b49fb7239dda admin admin] PATH_INFO: `/v3/projects/cfda065c9bdf4d8996649b48cf6b3285/users/f85d9a7d5a86499c8402b1c95d27518a/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:10.120145 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a4266b-260f-43ec-9d6f-b49fb7239dda admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfda065c9bdf4d8996649b48cf6b3285, user_id=f85d9a7d5a86499c8402b1c95d27518a, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:10.122717 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a4266b-260f-43ec-9d6f-b49fb7239dda admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:10.136107 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1234/2467] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:29:10 2022] PUT /identity/v3/projects/cfda065c9bdf4d8996649b48cf6b3285/users/f85d9a7d5a86499c8402b1c95d27518a/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:29:10.146353 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5ea03-8b49-43b8-8b00-3b83f67f2c59 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:10.146640 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5ea03-8b49-43b8-8b00-3b83f67f2c59 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:10.146943 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5ea03-8b49-43b8-8b00-3b83f67f2c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:10.173932 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-41a5ea03-8b49-43b8-8b00-3b83f67f2c59 None None] MFA Rules not processed for user `f85d9a7d5a86499c8402b1c95d27518a`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:29:10.175162 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-41a5ea03-8b49-43b8-8b00-3b83f67f2c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:29:10.237061 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1234/2468] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:29:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:29:10.248770 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4266b-260f-43ec-9d6f-b49fb7239dda admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:10.250442 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-16a4266b-260f-43ec-9d6f-b49fb7239dda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:29:10.306898 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4266b-260f-43ec-9d6f-b49fb7239dda admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:10.310479 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f612041c-e633-4258-9cbd-6cc83cf2866b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:10.311122 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f612041c-e633-4258-9cbd-6cc83cf2866b service cinder] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:10.311405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f612041c-e633-4258-9cbd-6cc83cf2866b service cinder] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:10.311673 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f612041c-e633-4258-9cbd-6cc83cf2866b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:10.319866 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f612041c-e633-4258-9cbd-6cc83cf2866b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:10.321298 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f612041c-e633-4258-9cbd-6cc83cf2866b service cinder] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:10.329975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1235/2469] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:29:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:29:20.234617 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=47,keystone:INSERT=3 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:29:20.325984 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:INSERT=2,keystone:SELECT=43 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:29:25.495280 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5ea03-8b49-43b8-8b00-3b83f67f2c59 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:25.496785 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-41a5ea03-8b49-43b8-8b00-3b83f67f2c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:29:25.584037 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5ea03-8b49-43b8-8b00-3b83f67f2c59 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:25.589263 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec3d612-8f3c-44d7-8834-350c89d588c5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18fe384ecd454fb298dcc788f41dd44b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:25.589901 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3d612-8f3c-44d7-8834-350c89d588c5 service placement] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:25.590173 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3d612-8f3c-44d7-8834-350c89d588c5 service placement] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:25.590444 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3d612-8f3c-44d7-8834-350c89d588c5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:25.592189 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-bec3d612-8f3c-44d7-8834-350c89d588c5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:29:25.603439 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec3d612-8f3c-44d7-8834-350c89d588c5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:25.605008 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec3d612-8f3c-44d7-8834-350c89d588c5 service placement] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:25.614642 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1235/2470] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:29:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:29:34.620527 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964b6bd-e44e-4fb4-8cba-c36a5d4e1eb8 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:34.621073 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964b6bd-e44e-4fb4-8cba-c36a5d4e1eb8 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:34.621179 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964b6bd-e44e-4fb4-8cba-c36a5d4e1eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:34.640048 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-3964b6bd-e44e-4fb4-8cba-c36a5d4e1eb8 None None] MFA Rules not processed for user `c3cf8e31062d4bd7bfa76c09e7e2adea`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:29:34.642000 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-3964b6bd-e44e-4fb4-8cba-c36a5d4e1eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:29:34.703232 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1236/2471] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:29:34 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:29:35.610581 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=32 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:29:44.696285 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=25 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:29:52.737853 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec3d612-8f3c-44d7-8834-350c89d588c5 service placement] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:52.739625 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-bec3d612-8f3c-44d7-8834-350c89d588c5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:29:52.804259 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec3d612-8f3c-44d7-8834-350c89d588c5 service placement] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:52.808757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52560b4-89bf-4e64-80b7-83088ecdf292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:52.809394 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52560b4-89bf-4e64-80b7-83088ecdf292 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:52.809671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52560b4-89bf-4e64-80b7-83088ecdf292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:52.809940 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52560b4-89bf-4e64-80b7-83088ecdf292 admin admin] PATH_INFO: `/v3/users/1891c1f56ceb4b9f9e90c358966a29a0` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:52.811094 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52560b4-89bf-4e64-80b7-83088ecdf292 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1891c1f56ceb4b9f9e90c358966a29a0)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:52.813024 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52560b4-89bf-4e64-80b7-83088ecdf292 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:52.865393 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-e52560b4-89bf-4e64-80b7-83088ecdf292 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1891c1f56ceb4b9f9e90c358966a29a0'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:29:52.866071 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-e52560b4-89bf-4e64-80b7-83088ecdf292 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1891c1f56ceb4b9f9e90c358966a29a0'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:29:52.866672 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-e52560b4-89bf-4e64-80b7-83088ecdf292 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1891c1f56ceb4b9f9e90c358966a29a0'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:29:52.879822 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-e52560b4-89bf-4e64-80b7-83088ecdf292 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1891c1f56ceb4b9f9e90c358966a29a0'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:29:52.887233 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-e52560b4-89bf-4e64-80b7-83088ecdf292 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1891c1f56ceb4b9f9e90c358966a29a0'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:29:52.903323 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1236/2472] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:29:52 2022] DELETE /identity/v3/users/1891c1f56ceb4b9f9e90c358966a29a0 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:29:53.074295 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964b6bd-e44e-4fb4-8cba-c36a5d4e1eb8 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:53.085984 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964b6bd-e44e-4fb4-8cba-c36a5d4e1eb8 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:53.142435 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cde02b-3ef3-48fa-9cfa-397174c8bc95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:53.143104 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cde02b-3ef3-48fa-9cfa-397174c8bc95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:53.143380 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cde02b-3ef3-48fa-9cfa-397174c8bc95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:53.143649 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cde02b-3ef3-48fa-9cfa-397174c8bc95 admin admin] PATH_INFO: `/v3/projects/cb04a85a9c074d1ba6a1fd2347fc5544` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:53.144750 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cde02b-3ef3-48fa-9cfa-397174c8bc95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb04a85a9c074d1ba6a1fd2347fc5544)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:53.147350 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cde02b-3ef3-48fa-9cfa-397174c8bc95 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:53.191578 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-f9cde02b-3ef3-48fa-9cfa-397174c8bc95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb04a85a9c074d1ba6a1fd2347fc5544', 'request_id': 'req-f9cde02b-3ef3-48fa-9cfa-397174c8bc95'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:29:53.198372 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-f9cde02b-3ef3-48fa-9cfa-397174c8bc95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb04a85a9c074d1ba6a1fd2347fc5544', 'request_id': 'req-f9cde02b-3ef3-48fa-9cfa-397174c8bc95'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:29:53.204343 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-f9cde02b-3ef3-48fa-9cfa-397174c8bc95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb04a85a9c074d1ba6a1fd2347fc5544', 'request_id': 'req-f9cde02b-3ef3-48fa-9cfa-397174c8bc95'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:29:53.216499 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-f9cde02b-3ef3-48fa-9cfa-397174c8bc95 admin admin] The token cache is being invalidate because project cb04a85a9c074d1ba6a1fd2347fc5544 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:29:53.216788 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-f9cde02b-3ef3-48fa-9cfa-397174c8bc95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:29:53.217431 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-f9cde02b-3ef3-48fa-9cfa-397174c8bc95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:29:53.218917 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1237/2473] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:29:53 2022] DELETE /identity/v3/projects/cb04a85a9c074d1ba6a1fd2347fc5544 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:29:53.222919 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52560b4-89bf-4e64-80b7-83088ecdf292 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:53.224404 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-e52560b4-89bf-4e64-80b7-83088ecdf292 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:29:53.236981 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52560b4-89bf-4e64-80b7-83088ecdf292 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:53.294410 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e81efa-7d44-4f83-8cb4-7712479b6d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:53.295074 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e81efa-7d44-4f83-8cb4-7712479b6d55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:53.295346 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e81efa-7d44-4f83-8cb4-7712479b6d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:53.295609 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e81efa-7d44-4f83-8cb4-7712479b6d55 admin admin] PATH_INFO: `/v3/users/9e3028d8dd8b4554a8ff4e1f9355a054` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:53.296754 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e81efa-7d44-4f83-8cb4-7712479b6d55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e3028d8dd8b4554a8ff4e1f9355a054)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:53.298638 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e81efa-7d44-4f83-8cb4-7712479b6d55 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:53.322563 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:29:53.348336 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-26e81efa-7d44-4f83-8cb4-7712479b6d55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e3028d8dd8b4554a8ff4e1f9355a054'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:29:53.348969 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-26e81efa-7d44-4f83-8cb4-7712479b6d55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e3028d8dd8b4554a8ff4e1f9355a054'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:29:53.349564 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-26e81efa-7d44-4f83-8cb4-7712479b6d55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e3028d8dd8b4554a8ff4e1f9355a054'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:29:53.365588 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-26e81efa-7d44-4f83-8cb4-7712479b6d55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e3028d8dd8b4554a8ff4e1f9355a054'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:29:53.372382 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-26e81efa-7d44-4f83-8cb4-7712479b6d55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e3028d8dd8b4554a8ff4e1f9355a054'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:29:53.385957 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1237/2474] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:29:53 2022] DELETE /identity/v3/users/9e3028d8dd8b4554a8ff4e1f9355a054 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:29:53.567753 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cde02b-3ef3-48fa-9cfa-397174c8bc95 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:53.569325 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-f9cde02b-3ef3-48fa-9cfa-397174c8bc95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:29:53.582592 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cde02b-3ef3-48fa-9cfa-397174c8bc95 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:53.640604 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e319e14e-b7a2-4031-8299-69b58ffb373f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:53.641230 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319e14e-b7a2-4031-8299-69b58ffb373f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:53.641510 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319e14e-b7a2-4031-8299-69b58ffb373f admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:53.641783 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319e14e-b7a2-4031-8299-69b58ffb373f admin admin] PATH_INFO: `/v3/projects/fa1307e6cbc2454bb1eea745990a3a53` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:53.642917 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e319e14e-b7a2-4031-8299-69b58ffb373f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa1307e6cbc2454bb1eea745990a3a53)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:53.644826 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e319e14e-b7a2-4031-8299-69b58ffb373f admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:53.678230 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:29:53.691593 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-e319e14e-b7a2-4031-8299-69b58ffb373f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa1307e6cbc2454bb1eea745990a3a53', 'request_id': 'req-e319e14e-b7a2-4031-8299-69b58ffb373f'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:29:53.698282 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-e319e14e-b7a2-4031-8299-69b58ffb373f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa1307e6cbc2454bb1eea745990a3a53', 'request_id': 'req-e319e14e-b7a2-4031-8299-69b58ffb373f'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:29:53.706852 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-e319e14e-b7a2-4031-8299-69b58ffb373f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa1307e6cbc2454bb1eea745990a3a53', 'request_id': 'req-e319e14e-b7a2-4031-8299-69b58ffb373f'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:29:53.721393 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-e319e14e-b7a2-4031-8299-69b58ffb373f admin admin] The token cache is being invalidate because project fa1307e6cbc2454bb1eea745990a3a53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:29:53.721686 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-e319e14e-b7a2-4031-8299-69b58ffb373f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:29:53.722526 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-e319e14e-b7a2-4031-8299-69b58ffb373f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:29:53.724248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1238/2475] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:29:53 2022] DELETE /identity/v3/projects/fa1307e6cbc2454bb1eea745990a3a53 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:29:53.729075 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd711d-df5d-4370-acdc-68894b50c3f5 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:53.729427 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd711d-df5d-4370-acdc-68894b50c3f5 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:53.729766 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd711d-df5d-4370-acdc-68894b50c3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:53.750128 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-a7bd711d-df5d-4370-acdc-68894b50c3f5 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:29:53.751350 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-a7bd711d-df5d-4370-acdc-68894b50c3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:29:53.813305 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1238/2476] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:29:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:29:53.821557 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183bcd9a-5d3d-4c95-aba1-f82a1fe5b234 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:53.821846 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183bcd9a-5d3d-4c95-aba1-f82a1fe5b234 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:53.822120 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183bcd9a-5d3d-4c95-aba1-f82a1fe5b234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:53.842281 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-183bcd9a-5d3d-4c95-aba1-f82a1fe5b234 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:29:53.843518 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-183bcd9a-5d3d-4c95-aba1-f82a1fe5b234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:29:53.851947 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1239/2477] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:29:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:29:53.856266 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd711d-df5d-4370-acdc-68894b50c3f5 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:53.866771 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd711d-df5d-4370-acdc-68894b50c3f5 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:53.871526 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae03b6df-f1d7-4972-ab72-7c56d65e3101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:53.872126 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae03b6df-f1d7-4972-ab72-7c56d65e3101 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:53.872438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae03b6df-f1d7-4972-ab72-7c56d65e3101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:53.872705 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae03b6df-f1d7-4972-ab72-7c56d65e3101 admin admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:53.873353 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae03b6df-f1d7-4972-ab72-7c56d65e3101 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:29:53.873657 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae03b6df-f1d7-4972-ab72-7c56d65e3101 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:53.875318 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae03b6df-f1d7-4972-ab72-7c56d65e3101 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:53.886890 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1239/2478] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:29:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:29:53.891238 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183bcd9a-5d3d-4c95-aba1-f82a1fe5b234 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:53.900580 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183bcd9a-5d3d-4c95-aba1-f82a1fe5b234 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:53.905263 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13f527-0dbb-41da-ba1d-c6bb7a4bbdfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:53.905857 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13f527-0dbb-41da-ba1d-c6bb7a4bbdfa admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:53.906134 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13f527-0dbb-41da-ba1d-c6bb7a4bbdfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:53.906402 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13f527-0dbb-41da-ba1d-c6bb7a4bbdfa admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:53.907349 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd13f527-0dbb-41da-ba1d-c6bb7a4bbdfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:53.909239 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd13f527-0dbb-41da-ba1d-c6bb7a4bbdfa admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:53.946430 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1240/2479] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:29:53 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:29:53.950484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae03b6df-f1d7-4972-ab72-7c56d65e3101 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:53.960309 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae03b6df-f1d7-4972-ab72-7c56d65e3101 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:54.016292 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a40a2d-064d-4aa3-8c02-6b58253e5a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:54.016937 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a40a2d-064d-4aa3-8c02-6b58253e5a01 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:54.017265 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a40a2d-064d-4aa3-8c02-6b58253e5a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:54.017542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a40a2d-064d-4aa3-8c02-6b58253e5a01 admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:54.018175 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a40a2d-064d-4aa3-8c02-6b58253e5a01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:54.020178 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a40a2d-064d-4aa3-8c02-6b58253e5a01 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:54.049228 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1240/2480] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:29:53 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:29:54.055543 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13f527-0dbb-41da-ba1d-c6bb7a4bbdfa admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:54.066370 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13f527-0dbb-41da-ba1d-c6bb7a4bbdfa admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:54.071269 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbac78e-289d-4674-afa5-fe2dc51831f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:54.071867 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbac78e-289d-4674-afa5-fe2dc51831f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:54.072143 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbac78e-289d-4674-afa5-fe2dc51831f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:54.072434 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbac78e-289d-4674-afa5-fe2dc51831f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:54.073086 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbac78e-289d-4674-afa5-fe2dc51831f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:54.075628 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbac78e-289d-4674-afa5-fe2dc51831f6 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:54.087318 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1241/2481] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:29:54.094334 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a40a2d-064d-4aa3-8c02-6b58253e5a01 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:54.110159 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a40a2d-064d-4aa3-8c02-6b58253e5a01 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:54.115710 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c45818-1bf8-4ec3-989d-a133cfcd3577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:54.116311 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c45818-1bf8-4ec3-989d-a133cfcd3577 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:54.116590 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c45818-1bf8-4ec3-989d-a133cfcd3577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:54.116863 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c45818-1bf8-4ec3-989d-a133cfcd3577 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:54.117505 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c45818-1bf8-4ec3-989d-a133cfcd3577 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:54.119146 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c45818-1bf8-4ec3-989d-a133cfcd3577 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:54.129886 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1241/2482] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:29:54.134153 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbac78e-289d-4674-afa5-fe2dc51831f6 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:54.144064 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbac78e-289d-4674-afa5-fe2dc51831f6 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:54.149133 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3890e9c-6966-42e3-886c-01eac4747b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:54.149771 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3890e9c-6966-42e3-886c-01eac4747b11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:54.150050 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3890e9c-6966-42e3-886c-01eac4747b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:54.150319 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3890e9c-6966-42e3-886c-01eac4747b11 admin admin] PATH_INFO: `/v3/projects/b5e2c167b38a436493bf8200c2baa07f/users/66b54fd5f231487c8f864414e7e9e39a/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:54.165739 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3890e9c-6966-42e3-886c-01eac4747b11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5e2c167b38a436493bf8200c2baa07f, user_id=66b54fd5f231487c8f864414e7e9e39a, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:54.168416 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3890e9c-6966-42e3-886c-01eac4747b11 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:54.181108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1242/2483] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:29:54 2022] PUT /identity/v3/projects/b5e2c167b38a436493bf8200c2baa07f/users/66b54fd5f231487c8f864414e7e9e39a/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:29:54.192499 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b74bf-5f05-4092-a384-3416e152ab68 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:54.192789 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b74bf-5f05-4092-a384-3416e152ab68 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:54.193062 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b74bf-5f05-4092-a384-3416e152ab68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:54.201067 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:29:54.222768 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-7a7b74bf-5f05-4092-a384-3416e152ab68 None None] MFA Rules not processed for user `66b54fd5f231487c8f864414e7e9e39a`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:29:54.224720 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-7a7b74bf-5f05-4092-a384-3416e152ab68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:29:54.287295 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1242/2484] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:29:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:29:54.297849 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3890e9c-6966-42e3-886c-01eac4747b11 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:54.299678 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-d3890e9c-6966-42e3-886c-01eac4747b11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:29:54.358720 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3890e9c-6966-42e3-886c-01eac4747b11 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:54.362547 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861bf846-197f-44f3-b68b-e41e29a881f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:54.363185 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861bf846-197f-44f3-b68b-e41e29a881f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:54.363465 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861bf846-197f-44f3-b68b-e41e29a881f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:54.363738 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861bf846-197f-44f3-b68b-e41e29a881f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:54.372167 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861bf846-197f-44f3-b68b-e41e29a881f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:54.373600 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861bf846-197f-44f3-b68b-e41e29a881f8 service cinder] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:54.382844 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1243/2485] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:29:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:29:56.450327 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99640172-6387-4c05-87bb-b2e8c2a7315a None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:56.450844 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99640172-6387-4c05-87bb-b2e8c2a7315a None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:56.450844 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99640172-6387-4c05-87bb-b2e8c2a7315a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:56.469272 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-99640172-6387-4c05-87bb-b2e8c2a7315a None None] MFA Rules not processed for user `d381a365e0744834b716fa6b0ea18d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:29:56.470447 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-99640172-6387-4c05-87bb-b2e8c2a7315a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:29:56.522959 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1243/2486] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:29:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:29:57.596878 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861bf846-197f-44f3-b68b-e41e29a881f8 service cinder] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:57.598512 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-861bf846-197f-44f3-b68b-e41e29a881f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:29:57.665644 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861bf846-197f-44f3-b68b-e41e29a881f8 service cinder] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:57.670176 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb8fc6-7d59-429c-ac5e-e93cf95ecf19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:57.670795 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb8fc6-7d59-429c-ac5e-e93cf95ecf19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:57.671106 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb8fc6-7d59-429c-ac5e-e93cf95ecf19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:57.671378 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb8fc6-7d59-429c-ac5e-e93cf95ecf19 admin admin] PATH_INFO: `/v3/users/f85d9a7d5a86499c8402b1c95d27518a` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:57.672516 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cb8fc6-7d59-429c-ac5e-e93cf95ecf19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f85d9a7d5a86499c8402b1c95d27518a)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:57.674409 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cb8fc6-7d59-429c-ac5e-e93cf95ecf19 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:57.675884 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:29:57.721670 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-13cb8fc6-7d59-429c-ac5e-e93cf95ecf19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f85d9a7d5a86499c8402b1c95d27518a'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:29:57.722237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-13cb8fc6-7d59-429c-ac5e-e93cf95ecf19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f85d9a7d5a86499c8402b1c95d27518a'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:29:57.729626 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-13cb8fc6-7d59-429c-ac5e-e93cf95ecf19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f85d9a7d5a86499c8402b1c95d27518a'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:29:57.730245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-13cb8fc6-7d59-429c-ac5e-e93cf95ecf19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f85d9a7d5a86499c8402b1c95d27518a'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:29:57.743084 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-13cb8fc6-7d59-429c-ac5e-e93cf95ecf19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f85d9a7d5a86499c8402b1c95d27518a'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:29:57.755297 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1244/2487] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:29:57 2022] DELETE /identity/v3/users/f85d9a7d5a86499c8402b1c95d27518a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:29:57.759695 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c91348-de82-4251-90d4-cfbb2386ab7e None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:57.760198 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c91348-de82-4251-90d4-cfbb2386ab7e None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:57.760291 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c91348-de82-4251-90d4-cfbb2386ab7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:57.778368 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-11c91348-de82-4251-90d4-cfbb2386ab7e None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:29:57.779645 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-11c91348-de82-4251-90d4-cfbb2386ab7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:29:57.836872 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1244/2488] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:29:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:29:57.848698 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb8fc6-7d59-429c-ac5e-e93cf95ecf19 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:57.850079 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-13cb8fc6-7d59-429c-ac5e-e93cf95ecf19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:29:57.907116 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb8fc6-7d59-429c-ac5e-e93cf95ecf19 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:57.911394 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76865a0a-f55f-4c15-81e2-10f51bdf34f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:57.911989 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76865a0a-f55f-4c15-81e2-10f51bdf34f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:57.912267 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76865a0a-f55f-4c15-81e2-10f51bdf34f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:57.912538 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76865a0a-f55f-4c15-81e2-10f51bdf34f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:57.921420 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76865a0a-f55f-4c15-81e2-10f51bdf34f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:57.922863 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76865a0a-f55f-4c15-81e2-10f51bdf34f8 service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:57.932218 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1245/2489] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:29:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:29:58.206466 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c91348-de82-4251-90d4-cfbb2386ab7e None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:58.208736 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-11c91348-de82-4251-90d4-cfbb2386ab7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:29:58.221046 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c91348-de82-4251-90d4-cfbb2386ab7e None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:58.226196 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32511251-a5d2-4a50-a800-4b4aae51e1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:58.227104 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32511251-a5d2-4a50-a800-4b4aae51e1ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:58.227386 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32511251-a5d2-4a50-a800-4b4aae51e1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:58.228073 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32511251-a5d2-4a50-a800-4b4aae51e1ba admin admin] PATH_INFO: `/v3/projects/cfda065c9bdf4d8996649b48cf6b3285` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:58.229207 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32511251-a5d2-4a50-a800-4b4aae51e1ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfda065c9bdf4d8996649b48cf6b3285)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:58.231149 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32511251-a5d2-4a50-a800-4b4aae51e1ba admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:58.272124 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:29:58.282671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-32511251-a5d2-4a50-a800-4b4aae51e1ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfda065c9bdf4d8996649b48cf6b3285', 'request_id': 'req-32511251-a5d2-4a50-a800-4b4aae51e1ba'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:29:58.289465 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-32511251-a5d2-4a50-a800-4b4aae51e1ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfda065c9bdf4d8996649b48cf6b3285', 'request_id': 'req-32511251-a5d2-4a50-a800-4b4aae51e1ba'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:29:58.295270 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-32511251-a5d2-4a50-a800-4b4aae51e1ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfda065c9bdf4d8996649b48cf6b3285', 'request_id': 'req-32511251-a5d2-4a50-a800-4b4aae51e1ba'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:29:58.307513 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-32511251-a5d2-4a50-a800-4b4aae51e1ba admin admin] The token cache is being invalidate because project cfda065c9bdf4d8996649b48cf6b3285 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:29:58.307868 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-32511251-a5d2-4a50-a800-4b4aae51e1ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:29:58.308530 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-32511251-a5d2-4a50-a800-4b4aae51e1ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:29:58.309985 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1245/2490] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:29:58 2022] DELETE /identity/v3/projects/cfda065c9bdf4d8996649b48cf6b3285 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:29:58.314825 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4cfbc-5a23-4eff-b922-c4fccd898fe8 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:58.315148 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4cfbc-5a23-4eff-b922-c4fccd898fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:58.315449 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4cfbc-5a23-4eff-b922-c4fccd898fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:58.335432 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-afd4cfbc-5a23-4eff-b922-c4fccd898fe8 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:29:58.336602 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-afd4cfbc-5a23-4eff-b922-c4fccd898fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:29:58.395934 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1246/2491] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:29:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:29:58.403595 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028c098-9487-418b-adce-23771e88ddd7 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:58.403931 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028c098-9487-418b-adce-23771e88ddd7 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:58.404204 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028c098-9487-418b-adce-23771e88ddd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:58.424772 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-b028c098-9487-418b-adce-23771e88ddd7 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:29:58.426027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-b028c098-9487-418b-adce-23771e88ddd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:29:58.434392 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1246/2492] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:29:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:29:58.438647 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd4cfbc-5a23-4eff-b922-c4fccd898fe8 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:58.448469 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd4cfbc-5a23-4eff-b922-c4fccd898fe8 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:58.453055 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8034b60-e21a-4ef1-9e6e-f3ceafb6f809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:58.453653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8034b60-e21a-4ef1-9e6e-f3ceafb6f809 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:58.453932 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8034b60-e21a-4ef1-9e6e-f3ceafb6f809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:58.454202 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8034b60-e21a-4ef1-9e6e-f3ceafb6f809 admin admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:58.454872 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8034b60-e21a-4ef1-9e6e-f3ceafb6f809 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:29:58.455179 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8034b60-e21a-4ef1-9e6e-f3ceafb6f809 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:58.456773 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8034b60-e21a-4ef1-9e6e-f3ceafb6f809 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:58.468644 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1247/2493] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:29:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:29:58.473471 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b028c098-9487-418b-adce-23771e88ddd7 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:58.483779 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b028c098-9487-418b-adce-23771e88ddd7 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:58.488480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e8052-7e72-4f57-828c-b9999dbe88ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:58.489569 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e8052-7e72-4f57-828c-b9999dbe88ed admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:58.489856 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e8052-7e72-4f57-828c-b9999dbe88ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:58.490117 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e8052-7e72-4f57-828c-b9999dbe88ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:58.490764 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959e8052-7e72-4f57-828c-b9999dbe88ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:58.492665 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959e8052-7e72-4f57-828c-b9999dbe88ed admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:58.523796 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1247/2494] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:29:58 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:29:58.525399 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8034b60-e21a-4ef1-9e6e-f3ceafb6f809 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:58.536027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8034b60-e21a-4ef1-9e6e-f3ceafb6f809 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:58.540884 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:29:58.595300 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2231cfde-4b51-447e-b0c1-bba5d97ccf26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:58.595937 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231cfde-4b51-447e-b0c1-bba5d97ccf26 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:58.596217 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231cfde-4b51-447e-b0c1-bba5d97ccf26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:58.596521 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231cfde-4b51-447e-b0c1-bba5d97ccf26 admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:58.597157 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2231cfde-4b51-447e-b0c1-bba5d97ccf26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:58.599678 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2231cfde-4b51-447e-b0c1-bba5d97ccf26 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:58.621168 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1248/2495] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:29:58 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:29:58.625218 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e8052-7e72-4f57-828c-b9999dbe88ed admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:58.635374 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e8052-7e72-4f57-828c-b9999dbe88ed admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:58.639865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0095014-35fd-4b58-9e03-e08841854744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:58.640460 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0095014-35fd-4b58-9e03-e08841854744 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:58.640732 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0095014-35fd-4b58-9e03-e08841854744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:58.640999 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0095014-35fd-4b58-9e03-e08841854744 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:58.641628 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0095014-35fd-4b58-9e03-e08841854744 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:58.643233 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0095014-35fd-4b58-9e03-e08841854744 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:58.653328 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1248/2496] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:29:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 14:29:58.657324 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2231cfde-4b51-447e-b0c1-bba5d97ccf26 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:58.667800 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2231cfde-4b51-447e-b0c1-bba5d97ccf26 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:58.672284 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc03e4a7-3952-4416-b101-65bf206d33e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:58.672866 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03e4a7-3952-4416-b101-65bf206d33e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:58.673141 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03e4a7-3952-4416-b101-65bf206d33e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:58.673408 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03e4a7-3952-4416-b101-65bf206d33e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:58.674040 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc03e4a7-3952-4416-b101-65bf206d33e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:58.675719 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc03e4a7-3952-4416-b101-65bf206d33e4 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:58.685965 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1249/2497] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:29:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:29:58.689808 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0095014-35fd-4b58-9e03-e08841854744 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:58.699405 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0095014-35fd-4b58-9e03-e08841854744 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:58.703854 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b26815c-2417-413a-bbbc-c4431527b35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:58.704479 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26815c-2417-413a-bbbc-c4431527b35b admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:58.704754 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26815c-2417-413a-bbbc-c4431527b35b admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:58.705023 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26815c-2417-413a-bbbc-c4431527b35b admin admin] PATH_INFO: `/v3/projects/5ab7af1a1c6749548a6ed520b119d868/users/e22e1db5255c4ac0903d94a382069db6/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:58.720421 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b26815c-2417-413a-bbbc-c4431527b35b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ab7af1a1c6749548a6ed520b119d868, user_id=e22e1db5255c4ac0903d94a382069db6, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:58.724070 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b26815c-2417-413a-bbbc-c4431527b35b admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:58.736946 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1249/2498] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:29:58 2022] PUT /identity/v3/projects/5ab7af1a1c6749548a6ed520b119d868/users/e22e1db5255c4ac0903d94a382069db6/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:29:58.741914 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31a0d6-319d-4127-afee-8674346dcbff None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:58.742197 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31a0d6-319d-4127-afee-8674346dcbff None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:58.742481 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31a0d6-319d-4127-afee-8674346dcbff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:58.760936 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-bc31a0d6-319d-4127-afee-8674346dcbff None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:29:58.762236 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-bc31a0d6-319d-4127-afee-8674346dcbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:29:58.819507 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1250/2499] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:29:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:29:58.829113 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b26815c-2417-413a-bbbc-c4431527b35b admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:58.830938 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-7b26815c-2417-413a-bbbc-c4431527b35b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:29:58.887470 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b26815c-2417-413a-bbbc-c4431527b35b admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:58.891147 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5984801-3a16-4342-8cd8-4cfc87050bb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:58.891756 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5984801-3a16-4342-8cd8-4cfc87050bb7 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:58.892032 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5984801-3a16-4342-8cd8-4cfc87050bb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:58.892301 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5984801-3a16-4342-8cd8-4cfc87050bb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:58.901594 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5984801-3a16-4342-8cd8-4cfc87050bb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:58.903045 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5984801-3a16-4342-8cd8-4cfc87050bb7 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:58.913097 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1250/2500] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:29:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:29:59.552671 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382c805-2a69-4964-b483-3a285c9b1ca0 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:59.552957 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382c805-2a69-4964-b483-3a285c9b1ca0 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:59.553234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382c805-2a69-4964-b483-3a285c9b1ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:59.583049 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-c382c805-2a69-4964-b483-3a285c9b1ca0 None None] MFA Rules not processed for user `e22e1db5255c4ac0903d94a382069db6`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:29:59.584266 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-c382c805-2a69-4964-b483-3a285c9b1ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:29:59.667196 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1251/2501] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:29:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:29:59.678049 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5984801-3a16-4342-8cd8-4cfc87050bb7 service neutron] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:29:59.679556 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-c5984801-3a16-4342-8cd8-4cfc87050bb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:29:59.749079 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5984801-3a16-4342-8cd8-4cfc87050bb7 service neutron] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:29:59.752469 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0093eacc-5f84-4958-acaf-5188cce36e6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:29:59.753072 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0093eacc-5f84-4958-acaf-5188cce36e6e service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:29:59.753348 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0093eacc-5f84-4958-acaf-5188cce36e6e service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:29:59.753619 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0093eacc-5f84-4958-acaf-5188cce36e6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:29:59.763107 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0093eacc-5f84-4958-acaf-5188cce36e6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:29:59.764523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0093eacc-5f84-4958-acaf-5188cce36e6e service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:29:59.773398 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1251/2502] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:29:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:30:03.241946 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ffe1c3-99e4-47f3-8ff8-f12e3b43e3d3 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:30:03.242481 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ffe1c3-99e4-47f3-8ff8-f12e3b43e3d3 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:30:03.242481 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ffe1c3-99e4-47f3-8ff8-f12e3b43e3d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:30:03.263643 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-c7ffe1c3-99e4-47f3-8ff8-f12e3b43e3d3 None None] MFA Rules not processed for user `e22e1db5255c4ac0903d94a382069db6`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:30:03.264974 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-c7ffe1c3-99e4-47f3-8ff8-f12e3b43e3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:30:03.274156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1252/2503] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:30:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:30:05.134197 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0093eacc-5f84-4958-acaf-5188cce36e6e service cinder] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:30:05.135672 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-0093eacc-5f84-4958-acaf-5188cce36e6e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:30:05.146193 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:30:05.151037 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0093eacc-5f84-4958-acaf-5188cce36e6e service cinder] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:30:05.157733 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f326b639-5c37-4a7e-aa57-536c41bdf49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:30:05.158544 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326b639-5c37-4a7e-aa57-536c41bdf49e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:30:05.159222 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326b639-5c37-4a7e-aa57-536c41bdf49e admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:30:05.159509 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326b639-5c37-4a7e-aa57-536c41bdf49e admin admin] PATH_INFO: `/v3/users/66b54fd5f231487c8f864414e7e9e39a` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:30:05.161248 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f326b639-5c37-4a7e-aa57-536c41bdf49e admin admin] RBAC: Authorizing `identity:delete_user(user_id=66b54fd5f231487c8f864414e7e9e39a)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:30:05.163265 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f326b639-5c37-4a7e-aa57-536c41bdf49e admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:30:05.208452 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-f326b639-5c37-4a7e-aa57-536c41bdf49e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66b54fd5f231487c8f864414e7e9e39a'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:30:05.209100 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-f326b639-5c37-4a7e-aa57-536c41bdf49e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66b54fd5f231487c8f864414e7e9e39a'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:30:05.209709 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-f326b639-5c37-4a7e-aa57-536c41bdf49e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66b54fd5f231487c8f864414e7e9e39a'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:30:05.221568 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-f326b639-5c37-4a7e-aa57-536c41bdf49e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66b54fd5f231487c8f864414e7e9e39a'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:30:05.228830 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-f326b639-5c37-4a7e-aa57-536c41bdf49e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66b54fd5f231487c8f864414e7e9e39a'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:30:05.241723 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1252/2504] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:30:05 2022] DELETE /identity/v3/users/66b54fd5f231487c8f864414e7e9e39a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:30:05.246194 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11bfd5c-afb3-455b-a77c-fa45b417f3cc None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:30:05.246674 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11bfd5c-afb3-455b-a77c-fa45b417f3cc None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:30:05.246769 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11bfd5c-afb3-455b-a77c-fa45b417f3cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:30:05.265312 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-d11bfd5c-afb3-455b-a77c-fa45b417f3cc None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:30:05.266490 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-d11bfd5c-afb3-455b-a77c-fa45b417f3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:30:05.271431 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:30:05.325062 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1253/2505] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:30:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:30:05.334802 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f326b639-5c37-4a7e-aa57-536c41bdf49e admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:30:05.336258 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-f326b639-5c37-4a7e-aa57-536c41bdf49e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:30:05.391760 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f326b639-5c37-4a7e-aa57-536c41bdf49e admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:30:05.395876 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0607bc54-ccfd-4b76-8ed6-e31eb10baf46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:30:05.396475 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607bc54-ccfd-4b76-8ed6-e31eb10baf46 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:30:05.396747 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607bc54-ccfd-4b76-8ed6-e31eb10baf46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:30:05.397017 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607bc54-ccfd-4b76-8ed6-e31eb10baf46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:30:05.405910 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0607bc54-ccfd-4b76-8ed6-e31eb10baf46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:30:05.407617 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0607bc54-ccfd-4b76-8ed6-e31eb10baf46 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:30:05.417339 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1253/2506] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:30:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:30:05.699752 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11bfd5c-afb3-455b-a77c-fa45b417f3cc None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:30:05.701170 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-d11bfd5c-afb3-455b-a77c-fa45b417f3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:30:05.713913 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11bfd5c-afb3-455b-a77c-fa45b417f3cc None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:30:05.718566 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f40df-c10a-41c5-a849-be3f47615880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:30:05.719282 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f40df-c10a-41c5-a849-be3f47615880 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:30:05.719559 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f40df-c10a-41c5-a849-be3f47615880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:30:05.719998 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f40df-c10a-41c5-a849-be3f47615880 admin admin] PATH_INFO: `/v3/projects/b5e2c167b38a436493bf8200c2baa07f` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:30:05.721302 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6f40df-c10a-41c5-a849-be3f47615880 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5e2c167b38a436493bf8200c2baa07f)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:30:05.723797 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6f40df-c10a-41c5-a849-be3f47615880 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:30:05.803257 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-4a6f40df-c10a-41c5-a849-be3f47615880 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5e2c167b38a436493bf8200c2baa07f', 'request_id': 'req-4a6f40df-c10a-41c5-a849-be3f47615880'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:30:05.811618 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-4a6f40df-c10a-41c5-a849-be3f47615880 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5e2c167b38a436493bf8200c2baa07f', 'request_id': 'req-4a6f40df-c10a-41c5-a849-be3f47615880'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:30:05.818129 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-4a6f40df-c10a-41c5-a849-be3f47615880 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5e2c167b38a436493bf8200c2baa07f', 'request_id': 'req-4a6f40df-c10a-41c5-a849-be3f47615880'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:30:05.836544 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-4a6f40df-c10a-41c5-a849-be3f47615880 admin admin] The token cache is being invalidate because project b5e2c167b38a436493bf8200c2baa07f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:30:05.836868 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-4a6f40df-c10a-41c5-a849-be3f47615880 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:30:05.837793 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-4a6f40df-c10a-41c5-a849-be3f47615880 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:30:05.839468 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1254/2507] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:30:05 2022] DELETE /identity/v3/projects/b5e2c167b38a436493bf8200c2baa07f => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:30:05.844541 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37311fde-292a-43ef-aadd-0422e00f1d11 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:30:05.844845 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37311fde-292a-43ef-aadd-0422e00f1d11 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:30:05.845106 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37311fde-292a-43ef-aadd-0422e00f1d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:30:05.864545 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-37311fde-292a-43ef-aadd-0422e00f1d11 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:30:05.865758 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-37311fde-292a-43ef-aadd-0422e00f1d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:30:05.931363 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1254/2508] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:30:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:30:05.945127 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5bbf3e-8a2a-433e-9b24-8e27ba829f5f None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:30:05.945405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5bbf3e-8a2a-433e-9b24-8e27ba829f5f None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:30:05.946246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5bbf3e-8a2a-433e-9b24-8e27ba829f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:30:05.967529 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-ff5bbf3e-8a2a-433e-9b24-8e27ba829f5f None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:30:05.968724 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-ff5bbf3e-8a2a-433e-9b24-8e27ba829f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:30:05.976783 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1255/2509] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:30:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:30:05.980960 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37311fde-292a-43ef-aadd-0422e00f1d11 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:30:05.991696 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37311fde-292a-43ef-aadd-0422e00f1d11 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:30:05.996292 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d0b13-d6c9-4243-9094-c75cd284a788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:30:05.996886 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d0b13-d6c9-4243-9094-c75cd284a788 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:30:05.997161 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d0b13-d6c9-4243-9094-c75cd284a788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:30:05.997432 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d0b13-d6c9-4243-9094-c75cd284a788 admin admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:30:05.998081 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d0b13-d6c9-4243-9094-c75cd284a788 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:30:05.998385 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d0b13-d6c9-4243-9094-c75cd284a788 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:30:06.000013 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d0b13-d6c9-4243-9094-c75cd284a788 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:30:06.017148 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1255/2510] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:30:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:30:06.017800 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5bbf3e-8a2a-433e-9b24-8e27ba829f5f None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:30:06.029621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5bbf3e-8a2a-433e-9b24-8e27ba829f5f None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:30:06.034603 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd133ab8-40fb-4f9c-8924-e3df0930e3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:30:06.035287 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd133ab8-40fb-4f9c-8924-e3df0930e3a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:30:06.035567 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd133ab8-40fb-4f9c-8924-e3df0930e3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:30:06.035835 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd133ab8-40fb-4f9c-8924-e3df0930e3a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:30:06.036481 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd133ab8-40fb-4f9c-8924-e3df0930e3a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:30:06.038393 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd133ab8-40fb-4f9c-8924-e3df0930e3a0 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:30:06.069926 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1256/2511] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:30:06 2022] POST /identity/v3/projects => generated 391 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:30:06.072049 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d0b13-d6c9-4243-9094-c75cd284a788 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:30:06.082620 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d0b13-d6c9-4243-9094-c75cd284a788 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:30:06.086139 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:30:06.148163 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b81e44-fcac-4cd9-b218-fd8a02b434d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:30:06.148843 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b81e44-fcac-4cd9-b218-fd8a02b434d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:30:06.149027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b81e44-fcac-4cd9-b218-fd8a02b434d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:30:06.149293 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b81e44-fcac-4cd9-b218-fd8a02b434d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:30:06.149915 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b81e44-fcac-4cd9-b218-fd8a02b434d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:30:06.151845 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b81e44-fcac-4cd9-b218-fd8a02b434d6 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:30:06.176386 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1256/2512] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:30:06 2022] POST /identity/v3/users => generated 296 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:30:06.177594 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd133ab8-40fb-4f9c-8924-e3df0930e3a0 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:30:06.188280 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd133ab8-40fb-4f9c-8924-e3df0930e3a0 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:30:06.193959 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f69a6b-aee3-4c98-b823-f899af60123a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:30:06.194557 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f69a6b-aee3-4c98-b823-f899af60123a admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:30:06.194862 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f69a6b-aee3-4c98-b823-f899af60123a admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:30:06.195138 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f69a6b-aee3-4c98-b823-f899af60123a admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:30:06.195772 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f69a6b-aee3-4c98-b823-f899af60123a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:30:06.197347 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f69a6b-aee3-4c98-b823-f899af60123a admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:30:06.207274 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1257/2513] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 14:30:06.211346 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b81e44-fcac-4cd9-b218-fd8a02b434d6 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:30:06.221228 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b81e44-fcac-4cd9-b218-fd8a02b434d6 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:30:06.225975 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811eec34-ce32-4d35-ae44-3e72c3a29b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:30:06.226576 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811eec34-ce32-4d35-ae44-3e72c3a29b67 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:30:06.226878 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811eec34-ce32-4d35-ae44-3e72c3a29b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:30:06.227153 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811eec34-ce32-4d35-ae44-3e72c3a29b67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:30:06.227793 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811eec34-ce32-4d35-ae44-3e72c3a29b67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:30:06.229384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811eec34-ce32-4d35-ae44-3e72c3a29b67 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:30:06.243961 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1257/2514] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:30:06.245152 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f69a6b-aee3-4c98-b823-f899af60123a admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:30:06.256038 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f69a6b-aee3-4c98-b823-f899af60123a admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:30:06.261895 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae616f21-a361-40cd-b94b-ca5d316cb705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:30:06.262560 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae616f21-a361-40cd-b94b-ca5d316cb705 admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:30:06.262873 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae616f21-a361-40cd-b94b-ca5d316cb705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:30:06.263138 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae616f21-a361-40cd-b94b-ca5d316cb705 admin admin] PATH_INFO: `/v3/projects/ca4deb7bc9b04bb28a027ee08495b164/users/009879bf69284a21aa431dcfaf2cc039/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:30:06.282857 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae616f21-a361-40cd-b94b-ca5d316cb705 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca4deb7bc9b04bb28a027ee08495b164, user_id=009879bf69284a21aa431dcfaf2cc039, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:30:06.285759 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae616f21-a361-40cd-b94b-ca5d316cb705 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:30:06.299683 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1258/2515] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:30:06 2022] PUT /identity/v3/projects/ca4deb7bc9b04bb28a027ee08495b164/users/009879bf69284a21aa431dcfaf2cc039/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:30:06.310377 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b394d2-6fdb-432a-9252-80b6fcb87e68 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:30:06.310661 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b394d2-6fdb-432a-9252-80b6fcb87e68 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:30:06.310973 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b394d2-6fdb-432a-9252-80b6fcb87e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:30:06.351414 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-23b394d2-6fdb-432a-9252-80b6fcb87e68 None None] MFA Rules not processed for user `009879bf69284a21aa431dcfaf2cc039`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:30:06.352635 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-23b394d2-6fdb-432a-9252-80b6fcb87e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:30:06.438859 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1258/2516] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:30:06 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:30:06.451032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae616f21-a361-40cd-b94b-ca5d316cb705 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:30:06.453776 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-ae616f21-a361-40cd-b94b-ca5d316cb705 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:30:06.490284 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:30:06.516781 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae616f21-a361-40cd-b94b-ca5d316cb705 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:30:06.521808 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4c009b-5f21-4fcd-b83a-2bd4236a3585 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:30:06.522573 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c009b-5f21-4fcd-b83a-2bd4236a3585 service cinder] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:30:06.522995 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c009b-5f21-4fcd-b83a-2bd4236a3585 service cinder] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:30:06.523370 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c009b-5f21-4fcd-b83a-2bd4236a3585 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:30:06.533526 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4c009b-5f21-4fcd-b83a-2bd4236a3585 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:30:06.535909 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4c009b-5f21-4fcd-b83a-2bd4236a3585 service cinder] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:30:06.545054 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1259/2517] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:30:06 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:30:16.435594 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=63,keystone:INSERT=3 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:30:16.540859 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=12 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:30:33.245838 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b394d2-6fdb-432a-9252-80b6fcb87e68 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:30:33.344841 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b394d2-6fdb-432a-9252-80b6fcb87e68 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:30:33.352237 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eec0e19-41a2-4501-893e-146645755ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:30:33.352682 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec0e19-41a2-4501-893e-146645755ba7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:30:33.352968 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec0e19-41a2-4501-893e-146645755ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:30:33.353267 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec0e19-41a2-4501-893e-146645755ba7 admin admin] PATH_INFO: `/v3/users/009879bf69284a21aa431dcfaf2cc039` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:30:33.354498 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eec0e19-41a2-4501-893e-146645755ba7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=009879bf69284a21aa431dcfaf2cc039)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:30:33.362229 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eec0e19-41a2-4501-893e-146645755ba7 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:30:33.436976 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-4eec0e19-41a2-4501-893e-146645755ba7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '009879bf69284a21aa431dcfaf2cc039'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:30:33.437783 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-4eec0e19-41a2-4501-893e-146645755ba7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '009879bf69284a21aa431dcfaf2cc039'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:30:33.438487 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-4eec0e19-41a2-4501-893e-146645755ba7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '009879bf69284a21aa431dcfaf2cc039'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:30:33.456573 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-4eec0e19-41a2-4501-893e-146645755ba7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '009879bf69284a21aa431dcfaf2cc039'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:30:33.465090 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-4eec0e19-41a2-4501-893e-146645755ba7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '009879bf69284a21aa431dcfaf2cc039'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:30:33.483800 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1259/2518] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:30:33 2022] DELETE /identity/v3/users/009879bf69284a21aa431dcfaf2cc039 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:30:33.488102 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b86fd-83f5-4bc1-b285-a9139a11e4f1 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:30:33.488865 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b86fd-83f5-4bc1-b285-a9139a11e4f1 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:30:33.489145 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b86fd-83f5-4bc1-b285-a9139a11e4f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:30:33.512540 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-989b86fd-83f5-4bc1-b285-a9139a11e4f1 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:30:33.513738 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-989b86fd-83f5-4bc1-b285-a9139a11e4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:30:33.580574 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1260/2519] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:30:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:30:33.593743 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eec0e19-41a2-4501-893e-146645755ba7 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:30:33.597023 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-4eec0e19-41a2-4501-893e-146645755ba7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:30:33.661524 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eec0e19-41a2-4501-893e-146645755ba7 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:30:33.665549 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf0778-d653-4dc3-8113-73dd3d80f491 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:30:33.666165 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf0778-d653-4dc3-8113-73dd3d80f491 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:30:33.666441 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf0778-d653-4dc3-8113-73dd3d80f491 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:30:33.666710 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf0778-d653-4dc3-8113-73dd3d80f491 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:30:33.676957 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf0778-d653-4dc3-8113-73dd3d80f491 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:30:33.678679 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf0778-d653-4dc3-8113-73dd3d80f491 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:30:33.696249 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1260/2520] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:30:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:30:33.991982 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b86fd-83f5-4bc1-b285-a9139a11e4f1 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:30:33.992925 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-989b86fd-83f5-4bc1-b285-a9139a11e4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:30:34.010196 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b86fd-83f5-4bc1-b285-a9139a11e4f1 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:30:34.015300 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133a42d-5498-48b7-8df2-6bcb23dfbe40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:30:34.015935 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133a42d-5498-48b7-8df2-6bcb23dfbe40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:30:34.016211 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133a42d-5498-48b7-8df2-6bcb23dfbe40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:30:34.017938 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133a42d-5498-48b7-8df2-6bcb23dfbe40 admin admin] PATH_INFO: `/v3/projects/ca4deb7bc9b04bb28a027ee08495b164` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:30:34.017938 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b133a42d-5498-48b7-8df2-6bcb23dfbe40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca4deb7bc9b04bb28a027ee08495b164)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:30:34.019653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b133a42d-5498-48b7-8df2-6bcb23dfbe40 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:30:34.083694 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-b133a42d-5498-48b7-8df2-6bcb23dfbe40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca4deb7bc9b04bb28a027ee08495b164', 'request_id': 'req-b133a42d-5498-48b7-8df2-6bcb23dfbe40'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:30:34.090112 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-b133a42d-5498-48b7-8df2-6bcb23dfbe40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca4deb7bc9b04bb28a027ee08495b164', 'request_id': 'req-b133a42d-5498-48b7-8df2-6bcb23dfbe40'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:30:34.097463 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-b133a42d-5498-48b7-8df2-6bcb23dfbe40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca4deb7bc9b04bb28a027ee08495b164', 'request_id': 'req-b133a42d-5498-48b7-8df2-6bcb23dfbe40'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:30:34.111644 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-b133a42d-5498-48b7-8df2-6bcb23dfbe40 admin admin] The token cache is being invalidate because project ca4deb7bc9b04bb28a027ee08495b164 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:30:34.112537 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-b133a42d-5498-48b7-8df2-6bcb23dfbe40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:30:34.113346 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-b133a42d-5498-48b7-8df2-6bcb23dfbe40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:30:34.115277 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1261/2521] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:30:33 2022] DELETE /identity/v3/projects/ca4deb7bc9b04bb28a027ee08495b164 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:30:43.206419 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a81842-d55e-4511-b0ad-a5378f7c9baf None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:30:43.206956 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a81842-d55e-4511-b0ad-a5378f7c9baf None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:30:43.207069 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a81842-d55e-4511-b0ad-a5378f7c9baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:30:43.225953 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-c9a81842-d55e-4511-b0ad-a5378f7c9baf None None] MFA Rules not processed for user `e22e1db5255c4ac0903d94a382069db6`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:30:43.227452 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-c9a81842-d55e-4511-b0ad-a5378f7c9baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:30:43.244493 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:30:43.292516 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1261/2522] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:30:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:30:44.107841 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:30:53.285336 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=15 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:31:06.656916 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133a42d-5498-48b7-8df2-6bcb23dfbe40 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:31:06.658383 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-b133a42d-5498-48b7-8df2-6bcb23dfbe40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:06.727427 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133a42d-5498-48b7-8df2-6bcb23dfbe40 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:31:06.731201 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45b553d-7f16-4f23-8cfe-44842961b72a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:31:06.732010 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b553d-7f16-4f23-8cfe-44842961b72a service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:06.732125 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b553d-7f16-4f23-8cfe-44842961b72a service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:06.732334 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b553d-7f16-4f23-8cfe-44842961b72a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:06.734073 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-c45b553d-7f16-4f23-8cfe-44842961b72a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:06.811761 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45b553d-7f16-4f23-8cfe-44842961b72a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:31:06.813205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45b553d-7f16-4f23-8cfe-44842961b72a service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:31:06.824260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1262/2523] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:31:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:31:08.008462 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a81842-d55e-4511-b0ad-a5378f7c9baf None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:31:08.009848 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-c9a81842-d55e-4511-b0ad-a5378f7c9baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:08.022015 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a81842-d55e-4511-b0ad-a5378f7c9baf None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:31:08.025536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952a3905-f37e-4533-81a7-93fcd7a8b272 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:31:08.026124 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a3905-f37e-4533-81a7-93fcd7a8b272 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:08.026400 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a3905-f37e-4533-81a7-93fcd7a8b272 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:08.026669 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a3905-f37e-4533-81a7-93fcd7a8b272 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:08.028320 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-952a3905-f37e-4533-81a7-93fcd7a8b272 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:08.041727 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952a3905-f37e-4533-81a7-93fcd7a8b272 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:31:08.043182 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952a3905-f37e-4533-81a7-93fcd7a8b272 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:31:08.054136 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1262/2524] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:31:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:31:16.819249 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=54 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:31:18.049619 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=6 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:31:21.186345 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cca339-6187-4b7e-92af-d0563643dc0b None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:21.186858 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cca339-6187-4b7e-92af-d0563643dc0b None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:21.186963 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cca339-6187-4b7e-92af-d0563643dc0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:21.206608 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-a6cca339-6187-4b7e-92af-d0563643dc0b None None] MFA Rules not processed for user `82328b7fd50649a485c6eaf27c5597c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:31:21.207705 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-a6cca339-6187-4b7e-92af-d0563643dc0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:21.263003 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1263/2525] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 14:31:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:31:21.308259 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952a3905-f37e-4533-81a7-93fcd7a8b272 service neutron] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:31:21.309800 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-952a3905-f37e-4533-81a7-93fcd7a8b272 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:21.383545 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952a3905-f37e-4533-81a7-93fcd7a8b272 service neutron] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:31:21.388843 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b618cf-3b26-44f2-856c-dd2883f45cf5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '879f296733114f6f844a98aaefe35e92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:31:21.389463 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b618cf-3b26-44f2-856c-dd2883f45cf5 service swift] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:21.390226 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b618cf-3b26-44f2-856c-dd2883f45cf5 service swift] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:21.390513 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b618cf-3b26-44f2-856c-dd2883f45cf5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:21.405430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b618cf-3b26-44f2-856c-dd2883f45cf5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:31:21.407751 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b618cf-3b26-44f2-856c-dd2883f45cf5 service swift] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:31:21.419706 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1263/2526] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 14:31:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 24 14:31:22.553200 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cca339-6187-4b7e-92af-d0563643dc0b None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:31:22.554717 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-a6cca339-6187-4b7e-92af-d0563643dc0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:22.571828 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cca339-6187-4b7e-92af-d0563643dc0b None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:31:22.579741 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0d4d6-796d-460e-a6b3-070988e07836 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:31:22.580746 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0d4d6-796d-460e-a6b3-070988e07836 service nova] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:22.581024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0d4d6-796d-460e-a6b3-070988e07836 service nova] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:22.581346 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0d4d6-796d-460e-a6b3-070988e07836 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:22.584219 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-fab0d4d6-796d-460e-a6b3-070988e07836 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:22.656027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab0d4d6-796d-460e-a6b3-070988e07836 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:31:22.657838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab0d4d6-796d-460e-a6b3-070988e07836 service nova] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:31:22.669742 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1264/2527] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:31:22 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:31:31.415397 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=32 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:31:32.665319 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=53 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:31:36.930494 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b618cf-3b26-44f2-856c-dd2883f45cf5 service swift] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:31:36.932637 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-82b618cf-3b26-44f2-856c-dd2883f45cf5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:37.019741 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b618cf-3b26-44f2-856c-dd2883f45cf5 service swift] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:31:37.026652 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af870a97-789f-4d7a-b699-8d1d7b4b92dc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18fe384ecd454fb298dcc788f41dd44b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:31:37.026652 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af870a97-789f-4d7a-b699-8d1d7b4b92dc service placement] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:37.026652 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af870a97-789f-4d7a-b699-8d1d7b4b92dc service placement] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:37.026652 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af870a97-789f-4d7a-b699-8d1d7b4b92dc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:37.028767 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0d4d6-796d-460e-a6b3-070988e07836 service nova] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:31:37.031098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-fab0d4d6-796d-460e-a6b3-070988e07836 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:37.033763 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-af870a97-789f-4d7a-b699-8d1d7b4b92dc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:37.051924 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af870a97-789f-4d7a-b699-8d1d7b4b92dc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:31:37.054259 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af870a97-789f-4d7a-b699-8d1d7b4b92dc service placement] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:31:37.068450 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1264/2528] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:31:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:31:37.126041 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0d4d6-796d-460e-a6b3-070988e07836 service nova] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:31:37.132791 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108692f5-265a-4b96-bff5-8adcf1093e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:31:37.133469 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108692f5-265a-4b96-bff5-8adcf1093e08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:37.133749 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108692f5-265a-4b96-bff5-8adcf1093e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:37.134036 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108692f5-265a-4b96-bff5-8adcf1093e08 admin admin] PATH_INFO: `/v3/users/d381a365e0744834b716fa6b0ea18d3d` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:37.135426 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108692f5-265a-4b96-bff5-8adcf1093e08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d381a365e0744834b716fa6b0ea18d3d)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:31:37.137354 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108692f5-265a-4b96-bff5-8adcf1093e08 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:31:37.199585 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-108692f5-265a-4b96-bff5-8adcf1093e08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd381a365e0744834b716fa6b0ea18d3d'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:31:37.202691 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-108692f5-265a-4b96-bff5-8adcf1093e08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd381a365e0744834b716fa6b0ea18d3d'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:31:37.213444 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-108692f5-265a-4b96-bff5-8adcf1093e08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd381a365e0744834b716fa6b0ea18d3d'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:31:37.214403 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-108692f5-265a-4b96-bff5-8adcf1093e08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd381a365e0744834b716fa6b0ea18d3d'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:31:37.238566 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-108692f5-265a-4b96-bff5-8adcf1093e08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd381a365e0744834b716fa6b0ea18d3d'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:31:37.256114 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1265/2529] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:31:37 2022] DELETE /identity/v3/users/d381a365e0744834b716fa6b0ea18d3d => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:31:37.464889 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af870a97-789f-4d7a-b699-8d1d7b4b92dc service placement] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:31:37.479024 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af870a97-789f-4d7a-b699-8d1d7b4b92dc service placement] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:31:37.554359 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef912e4-ccd4-4703-b828-c9d2172c35aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:31:37.555151 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef912e4-ccd4-4703-b828-c9d2172c35aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:37.555480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef912e4-ccd4-4703-b828-c9d2172c35aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:37.555758 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef912e4-ccd4-4703-b828-c9d2172c35aa admin admin] PATH_INFO: `/v3/projects/27e584f45faa47afa4ced9db997f4e4a` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:37.556878 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef912e4-ccd4-4703-b828-c9d2172c35aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=27e584f45faa47afa4ced9db997f4e4a)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:31:37.558928 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef912e4-ccd4-4703-b828-c9d2172c35aa admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:31:37.614830 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-bef912e4-ccd4-4703-b828-c9d2172c35aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27e584f45faa47afa4ced9db997f4e4a', 'request_id': 'req-bef912e4-ccd4-4703-b828-c9d2172c35aa'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:31:37.622050 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-bef912e4-ccd4-4703-b828-c9d2172c35aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27e584f45faa47afa4ced9db997f4e4a', 'request_id': 'req-bef912e4-ccd4-4703-b828-c9d2172c35aa'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:31:37.629681 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-bef912e4-ccd4-4703-b828-c9d2172c35aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27e584f45faa47afa4ced9db997f4e4a', 'request_id': 'req-bef912e4-ccd4-4703-b828-c9d2172c35aa'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:31:37.646430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-bef912e4-ccd4-4703-b828-c9d2172c35aa admin admin] The token cache is being invalidate because project 27e584f45faa47afa4ced9db997f4e4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:31:37.646856 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-bef912e4-ccd4-4703-b828-c9d2172c35aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:31:37.647568 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-bef912e4-ccd4-4703-b828-c9d2172c35aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:31:37.649330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1265/2530] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:31:37 2022] DELETE /identity/v3/projects/27e584f45faa47afa4ced9db997f4e4a => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:31:37.654695 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf7809-20a6-4aa8-9dfc-278eff3c44c9 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:37.655006 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf7809-20a6-4aa8-9dfc-278eff3c44c9 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:37.655282 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf7809-20a6-4aa8-9dfc-278eff3c44c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:37.681965 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-13cf7809-20a6-4aa8-9dfc-278eff3c44c9 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:31:37.683747 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-13cf7809-20a6-4aa8-9dfc-278eff3c44c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:37.774855 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1266/2531] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:31:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:31:37.833821 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43056329-68ff-4949-9f3c-8d0d461d3798 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:37.834134 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43056329-68ff-4949-9f3c-8d0d461d3798 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:37.834633 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43056329-68ff-4949-9f3c-8d0d461d3798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:37.855598 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-43056329-68ff-4949-9f3c-8d0d461d3798 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:31:37.856804 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-43056329-68ff-4949-9f3c-8d0d461d3798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:37.865323 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1266/2532] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:31:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:31:37.869805 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf7809-20a6-4aa8-9dfc-278eff3c44c9 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:31:37.880439 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf7809-20a6-4aa8-9dfc-278eff3c44c9 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:31:37.887182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976d02f-0562-4ec2-9dc2-7487791d7cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:31:37.887807 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976d02f-0562-4ec2-9dc2-7487791d7cc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:37.888088 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976d02f-0562-4ec2-9dc2-7487791d7cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:37.888367 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976d02f-0562-4ec2-9dc2-7487791d7cc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:37.889016 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d976d02f-0562-4ec2-9dc2-7487791d7cc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:31:37.889317 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d976d02f-0562-4ec2-9dc2-7487791d7cc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:31:37.890939 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d976d02f-0562-4ec2-9dc2-7487791d7cc0 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:31:37.907902 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1267/2533] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:31:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:31:37.909597 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43056329-68ff-4949-9f3c-8d0d461d3798 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:31:37.919422 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43056329-68ff-4949-9f3c-8d0d461d3798 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:31:37.925324 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0fd2b2-3053-4358-9fe0-a37f8f5c7d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:31:37.925946 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fd2b2-3053-4358-9fe0-a37f8f5c7d88 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:37.926240 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fd2b2-3053-4358-9fe0-a37f8f5c7d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:37.926521 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fd2b2-3053-4358-9fe0-a37f8f5c7d88 admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:37.927242 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0fd2b2-3053-4358-9fe0-a37f8f5c7d88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:31:37.929201 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0fd2b2-3053-4358-9fe0-a37f8f5c7d88 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:31:37.942449 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:31:37.965031 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1267/2534] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:31:37 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:31:37.969458 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976d02f-0562-4ec2-9dc2-7487791d7cc0 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:31:37.980936 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976d02f-0562-4ec2-9dc2-7487791d7cc0 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:31:37.984960 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:31:38.056060 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf0920-6b63-43ba-a74a-ea92e329e528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:31:38.057265 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf0920-6b63-43ba-a74a-ea92e329e528 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:38.057543 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf0920-6b63-43ba-a74a-ea92e329e528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:38.057812 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf0920-6b63-43ba-a74a-ea92e329e528 admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:38.058447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaf0920-6b63-43ba-a74a-ea92e329e528 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:31:38.060507 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaf0920-6b63-43ba-a74a-ea92e329e528 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:31:38.084102 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1268/2535] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:31:37 2022] POST /identity/v3/users => generated 292 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:31:38.088917 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0fd2b2-3053-4358-9fe0-a37f8f5c7d88 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:31:38.100884 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0fd2b2-3053-4358-9fe0-a37f8f5c7d88 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:31:38.106290 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c08587-08ea-46c4-9cfb-872bda7c7984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:31:38.106975 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c08587-08ea-46c4-9cfb-872bda7c7984 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:38.107275 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c08587-08ea-46c4-9cfb-872bda7c7984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:38.107583 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c08587-08ea-46c4-9cfb-872bda7c7984 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:38.108286 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c08587-08ea-46c4-9cfb-872bda7c7984 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:31:38.110007 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c08587-08ea-46c4-9cfb-872bda7c7984 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:31:38.125398 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1268/2536] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:31:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:31:38.127147 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf0920-6b63-43ba-a74a-ea92e329e528 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:31:38.139046 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf0920-6b63-43ba-a74a-ea92e329e528 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:31:38.144486 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32946371-3abc-46ef-8df3-217b90dee8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:31:38.145293 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32946371-3abc-46ef-8df3-217b90dee8a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:38.145728 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32946371-3abc-46ef-8df3-217b90dee8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:38.146116 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32946371-3abc-46ef-8df3-217b90dee8a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:38.146762 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32946371-3abc-46ef-8df3-217b90dee8a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:31:38.148462 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32946371-3abc-46ef-8df3-217b90dee8a2 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:31:38.162849 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1269/2537] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:31:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:31:38.166223 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c08587-08ea-46c4-9cfb-872bda7c7984 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:31:38.186722 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c08587-08ea-46c4-9cfb-872bda7c7984 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:31:38.193767 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c450fdcc-8049-417e-a43f-fc2154d930a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:31:38.195030 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c450fdcc-8049-417e-a43f-fc2154d930a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:38.195585 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c450fdcc-8049-417e-a43f-fc2154d930a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:38.196127 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c450fdcc-8049-417e-a43f-fc2154d930a6 admin admin] PATH_INFO: `/v3/projects/8830800be89540929716703bdbb07b13/users/27724ba162164bbb8faae54388ff11d4/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:38.217961 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c450fdcc-8049-417e-a43f-fc2154d930a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8830800be89540929716703bdbb07b13, user_id=27724ba162164bbb8faae54388ff11d4, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:31:38.223317 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c450fdcc-8049-417e-a43f-fc2154d930a6 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:31:38.242975 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1269/2538] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:31:38 2022] PUT /identity/v3/projects/8830800be89540929716703bdbb07b13/users/27724ba162164bbb8faae54388ff11d4/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:31:38.247765 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a872b28-4e65-4717-991d-1ac22b6a7224 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:38.248752 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a872b28-4e65-4717-991d-1ac22b6a7224 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:38.249260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a872b28-4e65-4717-991d-1ac22b6a7224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:38.274537 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-4a872b28-4e65-4717-991d-1ac22b6a7224 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:31:38.275878 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-4a872b28-4e65-4717-991d-1ac22b6a7224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:38.362249 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1270/2539] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:31:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:31:38.378200 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c450fdcc-8049-417e-a43f-fc2154d930a6 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:31:38.383068 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-c450fdcc-8049-417e-a43f-fc2154d930a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:38.481212 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c450fdcc-8049-417e-a43f-fc2154d930a6 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:31:38.485403 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f5fa29-d7c4-43f4-bb06-6ecfc6120635 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:31:38.486244 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5fa29-d7c4-43f4-bb06-6ecfc6120635 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:38.486698 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5fa29-d7c4-43f4-bb06-6ecfc6120635 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:38.487182 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5fa29-d7c4-43f4-bb06-6ecfc6120635 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:38.498031 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f5fa29-d7c4-43f4-bb06-6ecfc6120635 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:31:38.499890 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f5fa29-d7c4-43f4-bb06-6ecfc6120635 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:31:38.511945 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1270/2540] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:31:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:31:41.736750 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36953c-d40d-4ce5-b651-95308f2f289e None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:41.737292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36953c-d40d-4ce5-b651-95308f2f289e None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:41.737385 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36953c-d40d-4ce5-b651-95308f2f289e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:41.761659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-2f36953c-d40d-4ce5-b651-95308f2f289e None None] MFA Rules not processed for user `82328b7fd50649a485c6eaf27c5597c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:31:41.763079 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-2f36953c-d40d-4ce5-b651-95308f2f289e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:41.839245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1271/2541] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 14:31:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:31:41.853506 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f5fa29-d7c4-43f4-bb06-6ecfc6120635 service neutron] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:31:41.855825 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-29f5fa29-d7c4-43f4-bb06-6ecfc6120635 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:41.931085 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f5fa29-d7c4-43f4-bb06-6ecfc6120635 service neutron] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:31:41.936914 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e11fe2-5bba-421f-8bb4-d5cafd99a1f3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '879f296733114f6f844a98aaefe35e92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:31:41.937554 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e11fe2-5bba-421f-8bb4-d5cafd99a1f3 service swift] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:41.937819 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e11fe2-5bba-421f-8bb4-d5cafd99a1f3 service swift] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:41.938092 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e11fe2-5bba-421f-8bb4-d5cafd99a1f3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:41.949029 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e11fe2-5bba-421f-8bb4-d5cafd99a1f3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:31:41.950671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e11fe2-5bba-421f-8bb4-d5cafd99a1f3 service swift] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:31:41.960949 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1271/2542] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 14:31:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 14:31:43.008856 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a4f96-4a00-47a3-a4e8-2333d40f9948 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:43.009704 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a4f96-4a00-47a3-a4e8-2333d40f9948 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:43.010048 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a4f96-4a00-47a3-a4e8-2333d40f9948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:43.041028 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-f65a4f96-4a00-47a3-a4e8-2333d40f9948 None None] MFA Rules not processed for user `27724ba162164bbb8faae54388ff11d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:31:43.042920 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-f65a4f96-4a00-47a3-a4e8-2333d40f9948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:43.062157 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:31:43.112469 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1272/2543] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:31:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:31:43.116601 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e11fe2-5bba-421f-8bb4-d5cafd99a1f3 service swift] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:31:43.131862 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e11fe2-5bba-421f-8bb4-d5cafd99a1f3 service swift] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:31:43.136900 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba08d308-0419-4264-9539-280b2b8d6816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:31:43.137537 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08d308-0419-4264-9539-280b2b8d6816 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:43.137824 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08d308-0419-4264-9539-280b2b8d6816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:43.138089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08d308-0419-4264-9539-280b2b8d6816 admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:43.138749 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba08d308-0419-4264-9539-280b2b8d6816 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:31:43.140843 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba08d308-0419-4264-9539-280b2b8d6816 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:31:43.174085 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1272/2544] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:31:43 2022] POST /identity/v3/projects => generated 381 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:31:43.175285 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a4f96-4a00-47a3-a4e8-2333d40f9948 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:31:43.186715 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a4f96-4a00-47a3-a4e8-2333d40f9948 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:31:43.249296 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8dc0c4-60a0-4e65-9aa9-e92d087f9192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:31:43.249908 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8dc0c4-60a0-4e65-9aa9-e92d087f9192 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:43.250244 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8dc0c4-60a0-4e65-9aa9-e92d087f9192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:43.250519 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8dc0c4-60a0-4e65-9aa9-e92d087f9192 admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:43.251215 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8dc0c4-60a0-4e65-9aa9-e92d087f9192 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:31:43.253115 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8dc0c4-60a0-4e65-9aa9-e92d087f9192 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:31:43.274637 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1273/2545] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:31:43 2022] POST /identity/v3/users => generated 297 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:31:43.278914 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba08d308-0419-4264-9539-280b2b8d6816 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:31:43.315887 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba08d308-0419-4264-9539-280b2b8d6816 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:31:43.320958 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e507f4d2-9280-494b-b7ca-0b576ddd83b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:31:43.321574 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507f4d2-9280-494b-b7ca-0b576ddd83b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:43.321851 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507f4d2-9280-494b-b7ca-0b576ddd83b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:43.322179 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507f4d2-9280-494b-b7ca-0b576ddd83b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:43.322846 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e507f4d2-9280-494b-b7ca-0b576ddd83b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:31:43.324427 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e507f4d2-9280-494b-b7ca-0b576ddd83b9 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:31:43.336449 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1273/2546] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:31:43.341783 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8dc0c4-60a0-4e65-9aa9-e92d087f9192 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:31:43.354419 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8dc0c4-60a0-4e65-9aa9-e92d087f9192 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:31:43.359497 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec583b1-130e-4ef2-97ee-1e9cd6f60d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:31:43.360259 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec583b1-130e-4ef2-97ee-1e9cd6f60d26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:43.360535 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec583b1-130e-4ef2-97ee-1e9cd6f60d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:43.360805 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec583b1-130e-4ef2-97ee-1e9cd6f60d26 admin admin] PATH_INFO: `/v3/projects/e314f9937c1f46d88e3784e70f63a952/users/4bc2c1ce1de14353b6aa6e02785dfebd/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:43.377641 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec583b1-130e-4ef2-97ee-1e9cd6f60d26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e314f9937c1f46d88e3784e70f63a952, user_id=4bc2c1ce1de14353b6aa6e02785dfebd, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:31:43.380426 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec583b1-130e-4ef2-97ee-1e9cd6f60d26 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:31:43.395645 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1274/2547] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:31:43 2022] PUT /identity/v3/projects/e314f9937c1f46d88e3784e70f63a952/users/4bc2c1ce1de14353b6aa6e02785dfebd/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:31:43.399392 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e507f4d2-9280-494b-b7ca-0b576ddd83b9 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:31:43.410607 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e507f4d2-9280-494b-b7ca-0b576ddd83b9 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:31:43.429514 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:31:43.483330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced8ca2-c0aa-4b60-a8cd-77cf4f8c1ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:31:43.484016 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced8ca2-c0aa-4b60-a8cd-77cf4f8c1ebe admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:43.484262 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced8ca2-c0aa-4b60-a8cd-77cf4f8c1ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:43.484495 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced8ca2-c0aa-4b60-a8cd-77cf4f8c1ebe admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:43.485127 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ced8ca2-c0aa-4b60-a8cd-77cf4f8c1ebe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:31:43.487096 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ced8ca2-c0aa-4b60-a8cd-77cf4f8c1ebe admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:31:43.497610 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1274/2548] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:31:43.502109 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec583b1-130e-4ef2-97ee-1e9cd6f60d26 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:31:43.515581 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec583b1-130e-4ef2-97ee-1e9cd6f60d26 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:31:43.521033 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5404fd4-aa71-4686-a768-d3fb665f4e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:31:43.521706 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5404fd4-aa71-4686-a768-d3fb665f4e53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:43.521985 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5404fd4-aa71-4686-a768-d3fb665f4e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:43.522257 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5404fd4-aa71-4686-a768-d3fb665f4e53 admin admin] PATH_INFO: `/v3/projects/e314f9937c1f46d88e3784e70f63a952/users/4bc2c1ce1de14353b6aa6e02785dfebd/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:43.524670 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5404fd4-aa71-4686-a768-d3fb665f4e53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e314f9937c1f46d88e3784e70f63a952, user_id=4bc2c1ce1de14353b6aa6e02785dfebd, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:31:43.527378 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5404fd4-aa71-4686-a768-d3fb665f4e53 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:31:43.542212 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1275/2549] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:31:43 2022] PUT /identity/v3/projects/e314f9937c1f46d88e3784e70f63a952/users/4bc2c1ce1de14353b6aa6e02785dfebd/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:31:44.568527 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced8ca2-c0aa-4b60-a8cd-77cf4f8c1ebe admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:31:44.571288 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-7ced8ca2-c0aa-4b60-a8cd-77cf4f8c1ebe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:44.641287 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5404fd4-aa71-4686-a768-d3fb665f4e53 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:31:44.643171 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-a5404fd4-aa71-4686-a768-d3fb665f4e53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:44.733762 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced8ca2-c0aa-4b60-a8cd-77cf4f8c1ebe admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:31:44.749154 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5404fd4-aa71-4686-a768-d3fb665f4e53 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:31:44.756077 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd541c1-16f9-46c8-99d7-2c6bbf7eaebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:31:44.757020 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd541c1-16f9-46c8-99d7-2c6bbf7eaebd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:44.757490 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd541c1-16f9-46c8-99d7-2c6bbf7eaebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:44.757778 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd541c1-16f9-46c8-99d7-2c6bbf7eaebd admin admin] PATH_INFO: `/v3/users/e22e1db5255c4ac0903d94a382069db6` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:44.759329 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd541c1-16f9-46c8-99d7-2c6bbf7eaebd admin admin] RBAC: Authorizing `identity:delete_user(user_id=e22e1db5255c4ac0903d94a382069db6)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:31:44.761314 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd541c1-16f9-46c8-99d7-2c6bbf7eaebd admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:31:44.786782 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:31:44.860975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-fdd541c1-16f9-46c8-99d7-2c6bbf7eaebd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e22e1db5255c4ac0903d94a382069db6'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:31:44.865283 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-fdd541c1-16f9-46c8-99d7-2c6bbf7eaebd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e22e1db5255c4ac0903d94a382069db6'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:31:44.882600 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-fdd541c1-16f9-46c8-99d7-2c6bbf7eaebd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e22e1db5255c4ac0903d94a382069db6'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:31:44.884690 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-fdd541c1-16f9-46c8-99d7-2c6bbf7eaebd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e22e1db5255c4ac0903d94a382069db6'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:31:44.913608 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-fdd541c1-16f9-46c8-99d7-2c6bbf7eaebd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e22e1db5255c4ac0903d94a382069db6'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:31:44.930359 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b210d-954b-40a0-afcc-9eecbace7e0f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18fe384ecd454fb298dcc788f41dd44b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:31:44.931390 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b210d-954b-40a0-afcc-9eecbace7e0f service placement] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:44.932138 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b210d-954b-40a0-afcc-9eecbace7e0f service placement] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:44.932138 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b210d-954b-40a0-afcc-9eecbace7e0f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:44.939559 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-d85b210d-954b-40a0-afcc-9eecbace7e0f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:44.945050 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1276/2550] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:31:44 2022] DELETE /identity/v3/users/e22e1db5255c4ac0903d94a382069db6 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:31:44.953352 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85b210d-954b-40a0-afcc-9eecbace7e0f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:31:44.955105 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85b210d-954b-40a0-afcc-9eecbace7e0f service placement] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:31:44.970942 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1275/2551] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:31:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:31:45.349853 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd6a9d-4516-44af-ae06-08203ced92a5 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:45.350182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd6a9d-4516-44af-ae06-08203ced92a5 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:45.350294 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b210d-954b-40a0-afcc-9eecbace7e0f service placement] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:31:45.350503 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd6a9d-4516-44af-ae06-08203ced92a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:45.352024 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-d85b210d-954b-40a0-afcc-9eecbace7e0f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:45.373285 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-13cd6a9d-4516-44af-ae06-08203ced92a5 None None] MFA Rules not processed for user `82328b7fd50649a485c6eaf27c5597c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:31:45.374708 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-13cd6a9d-4516-44af-ae06-08203ced92a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:45.379941 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b210d-954b-40a0-afcc-9eecbace7e0f service placement] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:31:45.484067 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1277/2552] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 14:31:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:31:45.496131 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:31:45.496661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd6a9d-4516-44af-ae06-08203ced92a5 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:31:45.503183 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-13cd6a9d-4516-44af-ae06-08203ced92a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:45.527014 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4130953e-5063-447b-995c-0ee8c744a7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:31:45.530017 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130953e-5063-447b-995c-0ee8c744a7e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:45.530841 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130953e-5063-447b-995c-0ee8c744a7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:45.531355 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130953e-5063-447b-995c-0ee8c744a7e3 admin admin] PATH_INFO: `/v3/projects/5ab7af1a1c6749548a6ed520b119d868` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:45.535268 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4130953e-5063-447b-995c-0ee8c744a7e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ab7af1a1c6749548a6ed520b119d868)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:31:45.539858 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4130953e-5063-447b-995c-0ee8c744a7e3 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:31:45.624840 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd6a9d-4516-44af-ae06-08203ced92a5 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:31:45.631085 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-4130953e-5063-447b-995c-0ee8c744a7e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ab7af1a1c6749548a6ed520b119d868', 'request_id': 'req-4130953e-5063-447b-995c-0ee8c744a7e3'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:31:45.639753 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-4130953e-5063-447b-995c-0ee8c744a7e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ab7af1a1c6749548a6ed520b119d868', 'request_id': 'req-4130953e-5063-447b-995c-0ee8c744a7e3'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:31:45.647398 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-4130953e-5063-447b-995c-0ee8c744a7e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ab7af1a1c6749548a6ed520b119d868', 'request_id': 'req-4130953e-5063-447b-995c-0ee8c744a7e3'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:31:45.672668 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-4130953e-5063-447b-995c-0ee8c744a7e3 admin admin] The token cache is being invalidate because project 5ab7af1a1c6749548a6ed520b119d868 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:31:45.672964 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-4130953e-5063-447b-995c-0ee8c744a7e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:31:45.673629 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-4130953e-5063-447b-995c-0ee8c744a7e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:31:45.680048 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1276/2553] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:31:45 2022] DELETE /identity/v3/projects/5ab7af1a1c6749548a6ed520b119d868 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:31:45.681976 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40154cbf-5b1b-465f-a7a2-ce95232227d3 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:45.682255 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40154cbf-5b1b-465f-a7a2-ce95232227d3 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:45.682541 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40154cbf-5b1b-465f-a7a2-ce95232227d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:45.716047 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-40154cbf-5b1b-465f-a7a2-ce95232227d3 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:31:45.717463 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-40154cbf-5b1b-465f-a7a2-ce95232227d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:45.720730 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:31:45.804681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279f9f6a-bf14-48fb-adc3-3aae0599c311 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '879f296733114f6f844a98aaefe35e92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:31:45.805534 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f9f6a-bf14-48fb-adc3-3aae0599c311 service swift] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:45.805651 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f9f6a-bf14-48fb-adc3-3aae0599c311 service swift] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:45.805876 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f9f6a-bf14-48fb-adc3-3aae0599c311 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:45.808621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-279f9f6a-bf14-48fb-adc3-3aae0599c311 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:45.850330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1277/2554] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:31:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:31:45.852182 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ddaea6-0ae1-4219-b88a-f8370f855705 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:45.852682 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ddaea6-0ae1-4219-b88a-f8370f855705 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:45.853177 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ddaea6-0ae1-4219-b88a-f8370f855705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:45.899795 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-66ddaea6-0ae1-4219-b88a-f8370f855705 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:31:45.901181 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-66ddaea6-0ae1-4219-b88a-f8370f855705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:45.905889 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279f9f6a-bf14-48fb-adc3-3aae0599c311 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:31:45.908132 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279f9f6a-bf14-48fb-adc3-3aae0599c311 service swift] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:31:45.918003 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1278/2555] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 14:31:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 14:31:45.925963 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1278/2556] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:31:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:31:45.932533 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279f9f6a-bf14-48fb-adc3-3aae0599c311 service swift] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:31:45.952963 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279f9f6a-bf14-48fb-adc3-3aae0599c311 service swift] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:31:45.964234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd35e3-c321-4c30-a281-6b135f6cbdf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:31:45.964971 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd35e3-c321-4c30-a281-6b135f6cbdf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:45.965340 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd35e3-c321-4c30-a281-6b135f6cbdf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:45.965678 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd35e3-c321-4c30-a281-6b135f6cbdf8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:45.966396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fd35e3-c321-4c30-a281-6b135f6cbdf8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:31:45.966759 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fd35e3-c321-4c30-a281-6b135f6cbdf8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:31:45.968710 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fd35e3-c321-4c30-a281-6b135f6cbdf8 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:31:45.996373 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1279/2557] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:31:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:31:45.996764 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ddaea6-0ae1-4219-b88a-f8370f855705 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:31:46.007325 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ddaea6-0ae1-4219-b88a-f8370f855705 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:31:46.024661 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e955d-2d84-428c-8c8a-040e2db895ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:31:46.025371 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e955d-2d84-428c-8c8a-040e2db895ae admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:46.025795 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e955d-2d84-428c-8c8a-040e2db895ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:46.026142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e955d-2d84-428c-8c8a-040e2db895ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:46.026918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779e955d-2d84-428c-8c8a-040e2db895ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:31:46.030775 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779e955d-2d84-428c-8c8a-040e2db895ae admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:31:46.116666 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1279/2558] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:31:45 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:31:46.118988 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd35e3-c321-4c30-a281-6b135f6cbdf8 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:31:46.130614 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd35e3-c321-4c30-a281-6b135f6cbdf8 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:31:46.257869 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac671b-4ebe-4ed8-9856-460a24ad5036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:31:46.267720 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac671b-4ebe-4ed8-9856-460a24ad5036 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:46.268289 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac671b-4ebe-4ed8-9856-460a24ad5036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:46.268777 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac671b-4ebe-4ed8-9856-460a24ad5036 admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:46.269652 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ac671b-4ebe-4ed8-9856-460a24ad5036 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:31:46.281157 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ac671b-4ebe-4ed8-9856-460a24ad5036 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:31:46.333698 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1280/2559] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:31:46 2022] POST /identity/v3/users => generated 287 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:31:46.337133 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e955d-2d84-428c-8c8a-040e2db895ae admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:31:46.371447 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e955d-2d84-428c-8c8a-040e2db895ae admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:31:46.394275 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28433f5e-d569-4434-8308-3fa004024728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:31:46.395232 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28433f5e-d569-4434-8308-3fa004024728 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:46.400887 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28433f5e-d569-4434-8308-3fa004024728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:46.401216 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28433f5e-d569-4434-8308-3fa004024728 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:46.401932 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28433f5e-d569-4434-8308-3fa004024728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:31:46.410996 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28433f5e-d569-4434-8308-3fa004024728 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:31:46.440612 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1280/2560] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:31:46.442709 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac671b-4ebe-4ed8-9856-460a24ad5036 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:31:46.462465 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac671b-4ebe-4ed8-9856-460a24ad5036 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:31:46.473949 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc00a2b-078d-4b59-818a-77a4d5ec12f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:31:46.474862 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc00a2b-078d-4b59-818a-77a4d5ec12f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:46.475380 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc00a2b-078d-4b59-818a-77a4d5ec12f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:46.475887 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc00a2b-078d-4b59-818a-77a4d5ec12f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:46.477084 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc00a2b-078d-4b59-818a-77a4d5ec12f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:31:46.482039 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc00a2b-078d-4b59-818a-77a4d5ec12f3 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:31:46.515767 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1281/2561] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:31:46.520656 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28433f5e-d569-4434-8308-3fa004024728 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:31:46.533200 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28433f5e-d569-4434-8308-3fa004024728 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:31:46.545934 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c74ef-acfb-424f-ae8a-ec475f4c1875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:31:46.546646 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c74ef-acfb-424f-ae8a-ec475f4c1875 admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:46.546952 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c74ef-acfb-424f-ae8a-ec475f4c1875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:46.547226 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c74ef-acfb-424f-ae8a-ec475f4c1875 admin admin] PATH_INFO: `/v3/projects/119aafe09e994b0fb512e139c894c5e7/users/a15c0b58b74a476da6723e5478415be3/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:46.583331 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c74ef-acfb-424f-ae8a-ec475f4c1875 admin admin] RBAC: Authorizing `identity:create_grant(project_id=119aafe09e994b0fb512e139c894c5e7, user_id=a15c0b58b74a476da6723e5478415be3, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:31:46.586848 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c74ef-acfb-424f-ae8a-ec475f4c1875 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:31:46.625458 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1281/2562] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:31:46 2022] PUT /identity/v3/projects/119aafe09e994b0fb512e139c894c5e7/users/a15c0b58b74a476da6723e5478415be3/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:31:46.636001 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581bd77-8edf-48df-9fa6-12f402d1920d None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:46.636289 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581bd77-8edf-48df-9fa6-12f402d1920d None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:46.636559 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581bd77-8edf-48df-9fa6-12f402d1920d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:46.708471 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-4581bd77-8edf-48df-9fa6-12f402d1920d None None] MFA Rules not processed for user `a15c0b58b74a476da6723e5478415be3`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:31:46.709913 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-4581bd77-8edf-48df-9fa6-12f402d1920d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:46.786999 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:31:46.862747 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1282/2563] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:31:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:31:46.874571 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c74ef-acfb-424f-ae8a-ec475f4c1875 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:31:46.876787 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-b45c74ef-acfb-424f-ae8a-ec475f4c1875 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:46.904468 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:31:47.018448 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c74ef-acfb-424f-ae8a-ec475f4c1875 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:31:47.023649 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6050b151-6673-409b-9072-638ddcaf6044 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:31:47.024402 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6050b151-6673-409b-9072-638ddcaf6044 service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:47.024686 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6050b151-6673-409b-9072-638ddcaf6044 service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:47.024966 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6050b151-6673-409b-9072-638ddcaf6044 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:47.044792 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6050b151-6673-409b-9072-638ddcaf6044 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:31:47.046306 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6050b151-6673-409b-9072-638ddcaf6044 service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:31:47.079681 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1282/2564] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:31:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:31:48.700400 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b123aeea-480c-43de-bb43-0cfbb16d80c8 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:48.700938 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b123aeea-480c-43de-bb43-0cfbb16d80c8 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:48.700938 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b123aeea-480c-43de-bb43-0cfbb16d80c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:48.748306 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-b123aeea-480c-43de-bb43-0cfbb16d80c8 None None] MFA Rules not processed for user `a15c0b58b74a476da6723e5478415be3`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:31:48.755884 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-b123aeea-480c-43de-bb43-0cfbb16d80c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:48.771531 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1283/2565] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:31:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:31:49.716334 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921ec67-e848-4918-b256-30a370bd3a12 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:49.716911 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921ec67-e848-4918-b256-30a370bd3a12 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:49.716911 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921ec67-e848-4918-b256-30a370bd3a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:49.784302 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-3921ec67-e848-4918-b256-30a370bd3a12 None None] MFA Rules not processed for user `4bc2c1ce1de14353b6aa6e02785dfebd`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:31:49.785897 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-3921ec67-e848-4918-b256-30a370bd3a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:49.918025 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1283/2566] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:31:49 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:31:49.931729 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b123aeea-480c-43de-bb43-0cfbb16d80c8 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:31:49.933559 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-b123aeea-480c-43de-bb43-0cfbb16d80c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:49.993976 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3921ec67-e848-4918-b256-30a370bd3a12 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:31:49.995601 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-3921ec67-e848-4918-b256-30a370bd3a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:50.122107 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b123aeea-480c-43de-bb43-0cfbb16d80c8 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:31:50.129319 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4fe646-37da-4d8f-b1eb-2c2db776881d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e82b01dbd6e84a9d82baa2c28d77fc28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:31:50.130864 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4fe646-37da-4d8f-b1eb-2c2db776881d service glance] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:50.131320 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4fe646-37da-4d8f-b1eb-2c2db776881d service glance] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:50.131745 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4fe646-37da-4d8f-b1eb-2c2db776881d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:50.134733 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-1a4fe646-37da-4d8f-b1eb-2c2db776881d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:50.177561 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3921ec67-e848-4918-b256-30a370bd3a12 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:31:50.191461 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9ff8d-cb20-42a3-a13e-7d87d4a50ab6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18fe384ecd454fb298dcc788f41dd44b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:31:50.191848 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9ff8d-cb20-42a3-a13e-7d87d4a50ab6 service placement] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:50.192120 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9ff8d-cb20-42a3-a13e-7d87d4a50ab6 service placement] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:50.192391 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9ff8d-cb20-42a3-a13e-7d87d4a50ab6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:50.196630 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-50b9ff8d-cb20-42a3-a13e-7d87d4a50ab6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:50.213161 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b9ff8d-cb20-42a3-a13e-7d87d4a50ab6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:31:50.221137 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b9ff8d-cb20-42a3-a13e-7d87d4a50ab6 service placement] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:31:50.250910 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1284/2567] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:31:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:31:50.257455 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4fe646-37da-4d8f-b1eb-2c2db776881d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:31:50.258969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4fe646-37da-4d8f-b1eb-2c2db776881d service glance] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:31:50.283874 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1284/2568] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:31:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:31:50.526559 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f680f1-f755-44fa-961c-e9395f268493 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:50.526763 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f680f1-f755-44fa-961c-e9395f268493 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:50.527356 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f680f1-f755-44fa-961c-e9395f268493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:50.565543 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-26f680f1-f755-44fa-961c-e9395f268493 None None] MFA Rules not processed for user `82328b7fd50649a485c6eaf27c5597c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:31:50.568651 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-26f680f1-f755-44fa-961c-e9395f268493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:50.611015 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:31:50.792516 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1285/2569] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 14:31:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:31:50.853562 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4fe646-37da-4d8f-b1eb-2c2db776881d service glance] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:31:50.855945 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-1a4fe646-37da-4d8f-b1eb-2c2db776881d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:50.975944 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4fe646-37da-4d8f-b1eb-2c2db776881d service glance] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:31:50.984957 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa65f9-7859-4af8-aa28-e89b0368681b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '879f296733114f6f844a98aaefe35e92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:31:50.985821 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa65f9-7859-4af8-aa28-e89b0368681b service swift] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:50.986107 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa65f9-7859-4af8-aa28-e89b0368681b service swift] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:50.986380 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa65f9-7859-4af8-aa28-e89b0368681b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:51.001596 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aa65f9-7859-4af8-aa28-e89b0368681b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:31:51.004145 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aa65f9-7859-4af8-aa28-e89b0368681b service swift] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:31:51.017904 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1285/2570] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 14:31:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 14:31:54.989399 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f680f1-f755-44fa-961c-e9395f268493 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:31:54.991024 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-26f680f1-f755-44fa-961c-e9395f268493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:55.093075 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f680f1-f755-44fa-961c-e9395f268493 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:31:55.102114 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642e31c7-c6b0-4fd6-9191-0dfb44975063 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:31:55.103150 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e31c7-c6b0-4fd6-9191-0dfb44975063 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:55.103150 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e31c7-c6b0-4fd6-9191-0dfb44975063 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:55.103427 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e31c7-c6b0-4fd6-9191-0dfb44975063 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:55.107811 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-642e31c7-c6b0-4fd6-9191-0dfb44975063 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:55.130382 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642e31c7-c6b0-4fd6-9191-0dfb44975063 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:31:55.134392 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642e31c7-c6b0-4fd6-9191-0dfb44975063 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:31:55.158134 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1286/2571] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:31:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:31:56.570552 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa65f9-7859-4af8-aa28-e89b0368681b service swift] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:31:56.572975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-39aa65f9-7859-4af8-aa28-e89b0368681b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:56.616812 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:31:56.765434 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa65f9-7859-4af8-aa28-e89b0368681b service swift] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:31:56.772621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b04ae-7782-4bb7-9f35-8a3af5337e46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:31:56.773425 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b04ae-7782-4bb7-9f35-8a3af5337e46 service nova] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:31:56.773797 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b04ae-7782-4bb7-9f35-8a3af5337e46 service nova] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:31:56.774163 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b04ae-7782-4bb7-9f35-8a3af5337e46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:31:56.796142 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-4d3b04ae-7782-4bb7-9f35-8a3af5337e46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:31:56.833312 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3b04ae-7782-4bb7-9f35-8a3af5337e46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:31:56.835219 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3b04ae-7782-4bb7-9f35-8a3af5337e46 service nova] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:31:56.878851 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1286/2572] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:31:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:32:03.850594 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec0ffb8-adef-4251-9836-d1d6a3994de1 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:03.851206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec0ffb8-adef-4251-9836-d1d6a3994de1 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:03.851206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec0ffb8-adef-4251-9836-d1d6a3994de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:03.870119 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-7ec0ffb8-adef-4251-9836-d1d6a3994de1 None None] MFA Rules not processed for user `27724ba162164bbb8faae54388ff11d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:32:03.871345 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-7ec0ffb8-adef-4251-9836-d1d6a3994de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:32:03.878701 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1287/2573] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:32:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:32:04.130382 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b04ae-7782-4bb7-9f35-8a3af5337e46 service nova] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:32:04.131863 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-4d3b04ae-7782-4bb7-9f35-8a3af5337e46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:32:04.142991 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b04ae-7782-4bb7-9f35-8a3af5337e46 service nova] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:32:04.146463 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039cb5f4-3315-4a10-9d8b-ddcadfc0da88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:32:04.147108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039cb5f4-3315-4a10-9d8b-ddcadfc0da88 service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:04.147391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039cb5f4-3315-4a10-9d8b-ddcadfc0da88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:04.147644 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039cb5f4-3315-4a10-9d8b-ddcadfc0da88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:04.149273 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-039cb5f4-3315-4a10-9d8b-ddcadfc0da88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:32:04.158441 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039cb5f4-3315-4a10-9d8b-ddcadfc0da88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:32:04.159886 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039cb5f4-3315-4a10-9d8b-ddcadfc0da88 service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:32:04.167587 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1287/2574] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:32:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:32:08.605580 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec0ffb8-adef-4251-9836-d1d6a3994de1 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:32:08.607069 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-7ec0ffb8-adef-4251-9836-d1d6a3994de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:32:08.624050 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec0ffb8-adef-4251-9836-d1d6a3994de1 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:32:08.629623 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23427f0d-ad9c-4dd2-bca7-16872a2f8513 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:32:08.630229 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23427f0d-ad9c-4dd2-bca7-16872a2f8513 service nova] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:08.630508 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23427f0d-ad9c-4dd2-bca7-16872a2f8513 service nova] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:08.630776 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23427f0d-ad9c-4dd2-bca7-16872a2f8513 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:08.632573 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-23427f0d-ad9c-4dd2-bca7-16872a2f8513 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:32:08.645217 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23427f0d-ad9c-4dd2-bca7-16872a2f8513 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:32:08.647054 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23427f0d-ad9c-4dd2-bca7-16872a2f8513 service nova] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:32:08.659125 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1288/2575] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:32:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:32:09.166831 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fdb688-00ac-4047-b900-b74f291e6fa9 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:09.167369 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fdb688-00ac-4047-b900-b74f291e6fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:09.167478 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fdb688-00ac-4047-b900-b74f291e6fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:09.189855 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-16fdb688-00ac-4047-b900-b74f291e6fa9 None None] MFA Rules not processed for user `82328b7fd50649a485c6eaf27c5597c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:32:09.191201 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-16fdb688-00ac-4047-b900-b74f291e6fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:32:09.199613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1288/2576] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 14:32:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:32:09.215991 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23427f0d-ad9c-4dd2-bca7-16872a2f8513 service nova] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:32:09.229529 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23427f0d-ad9c-4dd2-bca7-16872a2f8513 service nova] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:32:09.236110 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd70025a-1112-4717-a3d3-d5ef2e8c9f63 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '879f296733114f6f844a98aaefe35e92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:32:09.236731 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70025a-1112-4717-a3d3-d5ef2e8c9f63 service swift] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:09.237011 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70025a-1112-4717-a3d3-d5ef2e8c9f63 service swift] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:09.237346 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70025a-1112-4717-a3d3-d5ef2e8c9f63 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:09.247113 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd70025a-1112-4717-a3d3-d5ef2e8c9f63 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:32:09.249350 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd70025a-1112-4717-a3d3-d5ef2e8c9f63 service swift] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:32:09.263437 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1289/2577] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 14:32:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 14:32:10.540609 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918a164-a0e0-46fa-816d-5079736d9298 None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:10.541146 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918a164-a0e0-46fa-816d-5079736d9298 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:10.541146 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918a164-a0e0-46fa-816d-5079736d9298 None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:10.542490 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1289/2578] 10.222.0.52 () {54 vars in 921 bytes} [Sat Sep 24 14:32:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:32:10.549293 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae646a9-f645-473d-88ca-f7577db19d79 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:10.549825 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae646a9-f645-473d-88ca-f7577db19d79 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:10.549825 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae646a9-f645-473d-88ca-f7577db19d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:10.584035 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-bae646a9-f645-473d-88ca-f7577db19d79 None None] MFA Rules not processed for user `79eaaae97aac4282a613939e5b768376`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:32:10.585292 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-bae646a9-f645-473d-88ca-f7577db19d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:32:10.608981 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1290/2579] 10.222.0.52 () {58 vars in 1018 bytes} [Sat Sep 24 14:32:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:32:10.624359 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918a164-a0e0-46fa-816d-5079736d9298 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:32:10.626309 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-2918a164-a0e0-46fa-816d-5079736d9298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:32:10.656428 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918a164-a0e0-46fa-816d-5079736d9298 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:32:10.664061 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09588d4-2b99-4348-83e0-34d6e57545ae service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e82b01dbd6e84a9d82baa2c28d77fc28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:32:10.665070 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09588d4-2b99-4348-83e0-34d6e57545ae service glance] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:10.665605 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09588d4-2b99-4348-83e0-34d6e57545ae service glance] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:10.666138 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09588d4-2b99-4348-83e0-34d6e57545ae service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:10.712168 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09588d4-2b99-4348-83e0-34d6e57545ae service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:32:10.714225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09588d4-2b99-4348-83e0-34d6e57545ae service glance] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:32:10.726031 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1290/2580] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:32:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:32:10.864651 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a733c-89ac-42b4-afa3-d09d6c925718 None None] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:10.865445 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a733c-89ac-42b4-afa3-d09d6c925718 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:10.865979 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a733c-89ac-42b4-afa3-d09d6c925718 None None] PATH_INFO: `/` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:10.868702 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1291/2581] 10.222.0.52 () {54 vars in 921 bytes} [Sat Sep 24 14:32:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:32:10.873638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b9ba3-8dd5-4949-a747-203c47aff3bd None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:10.873920 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b9ba3-8dd5-4949-a747-203c47aff3bd None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:10.875040 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b9ba3-8dd5-4949-a747-203c47aff3bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:10.896239 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-9d4b9ba3-8dd5-4949-a747-203c47aff3bd None None] MFA Rules not processed for user `a9348164132846d2b593ddfa29a820eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:32:10.897529 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-9d4b9ba3-8dd5-4949-a747-203c47aff3bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:32:10.908201 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1291/2582] 10.222.0.52 () {58 vars in 1018 bytes} [Sat Sep 24 14:32:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:32:10.920297 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a733c-89ac-42b4-afa3-d09d6c925718 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:32:10.934251 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a733c-89ac-42b4-afa3-d09d6c925718 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:32:10.938355 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5fa78c-e63e-43f8-b990-8d9d9f6e7fc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:32:10.939546 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5fa78c-e63e-43f8-b990-8d9d9f6e7fc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:10.940199 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5fa78c-e63e-43f8-b990-8d9d9f6e7fc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:10.940678 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5fa78c-e63e-43f8-b990-8d9d9f6e7fc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:10.950510 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5fa78c-e63e-43f8-b990-8d9d9f6e7fc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:32:10.952013 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5fa78c-e63e-43f8-b990-8d9d9f6e7fc6 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:32:10.961515 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1292/2583] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:32:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:32:11.062423 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8c425-0df4-46af-a1cb-a32708a9b4dc None None] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:11.062878 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8c425-0df4-46af-a1cb-a32708a9b4dc None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:11.063256 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8c425-0df4-46af-a1cb-a32708a9b4dc None None] PATH_INFO: `/` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:11.064708 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1292/2584] 10.222.0.52 () {54 vars in 921 bytes} [Sat Sep 24 14:32:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 14:32:11.070036 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b6d92-8cc6-4418-a860-d7e4dd13eb17 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:11.070325 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b6d92-8cc6-4418-a860-d7e4dd13eb17 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:11.070599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b6d92-8cc6-4418-a860-d7e4dd13eb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:11.092071 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-a82b6d92-8cc6-4418-a860-d7e4dd13eb17 None None] MFA Rules not processed for user `79eaaae97aac4282a613939e5b768376`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:32:11.093558 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-a82b6d92-8cc6-4418-a860-d7e4dd13eb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:32:11.103602 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1293/2585] 10.222.0.52 () {58 vars in 1018 bytes} [Sat Sep 24 14:32:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:32:11.114861 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa8c425-0df4-46af-a1cb-a32708a9b4dc None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:32:11.127160 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa8c425-0df4-46af-a1cb-a32708a9b4dc None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:32:11.130743 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcea916-6c75-49eb-a98a-d062e3408bf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:32:11.131423 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcea916-6c75-49eb-a98a-d062e3408bf7 service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:11.131705 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcea916-6c75-49eb-a98a-d062e3408bf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:11.131978 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcea916-6c75-49eb-a98a-d062e3408bf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:11.143406 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcea916-6c75-49eb-a98a-d062e3408bf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:32:11.144864 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcea916-6c75-49eb-a98a-d062e3408bf7 service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:32:11.155801 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1293/2586] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:32:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 14:32:21.085752 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=65 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:32:21.150726 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=52 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:32:29.271975 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82b6d92-8cc6-4418-a860-d7e4dd13eb17 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:32:29.283894 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82b6d92-8cc6-4418-a860-d7e4dd13eb17 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:32:29.289409 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2867dc-d9c5-494f-af69-5542da61eb8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:32:29.290014 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2867dc-d9c5-494f-af69-5542da61eb8e service nova] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:29.290296 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2867dc-d9c5-494f-af69-5542da61eb8e service nova] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:29.290563 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2867dc-d9c5-494f-af69-5542da61eb8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:29.292339 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-fd2867dc-d9c5-494f-af69-5542da61eb8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:32:29.350642 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2867dc-d9c5-494f-af69-5542da61eb8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:32:29.352260 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2867dc-d9c5-494f-af69-5542da61eb8e service nova] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:32:29.361632 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1294/2587] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:32:29 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:32:32.713476 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3d76e-2964-46b4-b995-ab8c87bd95fb None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:32.714096 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3d76e-2964-46b4-b995-ab8c87bd95fb None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:32.714282 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3d76e-2964-46b4-b995-ab8c87bd95fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:32.736508 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-83c3d76e-2964-46b4-b995-ab8c87bd95fb None None] MFA Rules not processed for user `82328b7fd50649a485c6eaf27c5597c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:32:32.737852 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-83c3d76e-2964-46b4-b995-ab8c87bd95fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:32:32.746969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1294/2588] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 14:32:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:32:32.773014 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2867dc-d9c5-494f-af69-5542da61eb8e service nova] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:32:32.785578 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2867dc-d9c5-494f-af69-5542da61eb8e service nova] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:32:32.790757 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce083612-c768-4654-8b29-16bc913f82dd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '879f296733114f6f844a98aaefe35e92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:32:32.791528 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce083612-c768-4654-8b29-16bc913f82dd service swift] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:32.791812 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce083612-c768-4654-8b29-16bc913f82dd service swift] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:32.792082 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce083612-c768-4654-8b29-16bc913f82dd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:32.801203 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce083612-c768-4654-8b29-16bc913f82dd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:32:32.802775 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce083612-c768-4654-8b29-16bc913f82dd service swift] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:32:32.811760 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1295/2589] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 14:32:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 14:32:42.730967 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=3 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:32:42.808074 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=34 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:32:47.461852 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5838a3d2-e5f1-40bf-a041-153a696bfa51 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:47.462327 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5838a3d2-e5f1-40bf-a041-153a696bfa51 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:47.462616 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5838a3d2-e5f1-40bf-a041-153a696bfa51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:47.483442 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-5838a3d2-e5f1-40bf-a041-153a696bfa51 None None] MFA Rules not processed for user `82328b7fd50649a485c6eaf27c5597c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:32:47.484699 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-5838a3d2-e5f1-40bf-a041-153a696bfa51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:32:47.497981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1295/2590] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 14:32:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:32:47.507574 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce083612-c768-4654-8b29-16bc913f82dd service swift] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:32:47.518373 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce083612-c768-4654-8b29-16bc913f82dd service swift] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:32:47.523375 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6973253-32be-4237-99e4-4ce84cd11217 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '879f296733114f6f844a98aaefe35e92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:32:47.523974 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6973253-32be-4237-99e4-4ce84cd11217 service swift] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:47.524253 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6973253-32be-4237-99e4-4ce84cd11217 service swift] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:47.524547 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6973253-32be-4237-99e4-4ce84cd11217 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:47.532996 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6973253-32be-4237-99e4-4ce84cd11217 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:32:47.534567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6973253-32be-4237-99e4-4ce84cd11217 service swift] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:32:47.542919 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1296/2591] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 14:32:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 14:32:48.058336 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370af895-f5d6-4290-95b9-c211ed24b91b None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:48.058621 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370af895-f5d6-4290-95b9-c211ed24b91b None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:48.058931 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370af895-f5d6-4290-95b9-c211ed24b91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:48.078225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-370af895-f5d6-4290-95b9-c211ed24b91b None None] MFA Rules not processed for user `82328b7fd50649a485c6eaf27c5597c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:32:48.079501 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-370af895-f5d6-4290-95b9-c211ed24b91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:32:48.086526 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1296/2592] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 14:32:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:32:48.099105 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6973253-32be-4237-99e4-4ce84cd11217 service swift] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:32:48.109625 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6973253-32be-4237-99e4-4ce84cd11217 service swift] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:32:48.115130 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfbef46-90ad-43a6-82fc-eb06b22e9361 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '879f296733114f6f844a98aaefe35e92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:32:48.115720 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbef46-90ad-43a6-82fc-eb06b22e9361 service swift] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:48.116002 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbef46-90ad-43a6-82fc-eb06b22e9361 service swift] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:48.116278 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbef46-90ad-43a6-82fc-eb06b22e9361 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:48.124117 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfbef46-90ad-43a6-82fc-eb06b22e9361 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:32:48.125765 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfbef46-90ad-43a6-82fc-eb06b22e9361 service swift] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:32:48.133902 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1297/2593] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 14:32:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 14:32:48.374517 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8dd83-d543-4ab6-9c5b-1b13551df4b3 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:48.374838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8dd83-d543-4ab6-9c5b-1b13551df4b3 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:48.375122 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8dd83-d543-4ab6-9c5b-1b13551df4b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:48.396650 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-24a8dd83-d543-4ab6-9c5b-1b13551df4b3 None None] MFA Rules not processed for user `82328b7fd50649a485c6eaf27c5597c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:32:48.397961 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-24a8dd83-d543-4ab6-9c5b-1b13551df4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:32:48.405600 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1297/2594] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 14:32:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:32:48.419078 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfbef46-90ad-43a6-82fc-eb06b22e9361 service swift] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:32:48.429923 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfbef46-90ad-43a6-82fc-eb06b22e9361 service swift] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:32:48.435049 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f335fc31-839b-4d2b-8a72-c890a7f0c13b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '879f296733114f6f844a98aaefe35e92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:32:48.435641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f335fc31-839b-4d2b-8a72-c890a7f0c13b service swift] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:48.435920 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f335fc31-839b-4d2b-8a72-c890a7f0c13b service swift] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:48.436194 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f335fc31-839b-4d2b-8a72-c890a7f0c13b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:48.445569 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f335fc31-839b-4d2b-8a72-c890a7f0c13b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:32:48.447267 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f335fc31-839b-4d2b-8a72-c890a7f0c13b service swift] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:32:48.455751 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1298/2595] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 14:32:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 14:32:48.825916 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cebcd9-aa1a-4a64-9b3d-023a9823c85a None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:48.826386 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cebcd9-aa1a-4a64-9b3d-023a9823c85a None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:48.826484 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cebcd9-aa1a-4a64-9b3d-023a9823c85a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:48.849267 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-b7cebcd9-aa1a-4a64-9b3d-023a9823c85a None None] MFA Rules not processed for user `82328b7fd50649a485c6eaf27c5597c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:32:48.850487 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-b7cebcd9-aa1a-4a64-9b3d-023a9823c85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:32:48.857672 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1298/2596] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 14:32:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:32:48.870075 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f335fc31-839b-4d2b-8a72-c890a7f0c13b service swift] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:32:48.880593 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f335fc31-839b-4d2b-8a72-c890a7f0c13b service swift] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:32:48.885816 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e92a398-ed60-4a26-b58d-899a0efc94a2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '879f296733114f6f844a98aaefe35e92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:32:48.886718 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92a398-ed60-4a26-b58d-899a0efc94a2 service swift] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:48.887042 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92a398-ed60-4a26-b58d-899a0efc94a2 service swift] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:48.887316 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92a398-ed60-4a26-b58d-899a0efc94a2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:48.896659 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e92a398-ed60-4a26-b58d-899a0efc94a2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:32:48.898318 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e92a398-ed60-4a26-b58d-899a0efc94a2 service swift] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:32:48.907443 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1299/2597] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 14:32:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 14:32:51.067654 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cebcd9-aa1a-4a64-9b3d-023a9823c85a None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:32:51.140859 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cebcd9-aa1a-4a64-9b3d-023a9823c85a None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:32:51.145736 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c31f1-a7b5-4a58-acb1-3415db68eb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:32:51.146361 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c31f1-a7b5-4a58-acb1-3415db68eb06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:51.146698 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c31f1-a7b5-4a58-acb1-3415db68eb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:51.146996 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c31f1-a7b5-4a58-acb1-3415db68eb06 admin admin] PATH_INFO: `/v3/users/a15c0b58b74a476da6723e5478415be3` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:51.148117 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4c31f1-a7b5-4a58-acb1-3415db68eb06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a15c0b58b74a476da6723e5478415be3)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:32:51.150108 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4c31f1-a7b5-4a58-acb1-3415db68eb06 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:32:51.197307 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-fd4c31f1-a7b5-4a58-acb1-3415db68eb06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a15c0b58b74a476da6723e5478415be3'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:32:51.197960 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-fd4c31f1-a7b5-4a58-acb1-3415db68eb06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a15c0b58b74a476da6723e5478415be3'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:32:51.209548 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-fd4c31f1-a7b5-4a58-acb1-3415db68eb06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a15c0b58b74a476da6723e5478415be3'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:32:51.210268 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-fd4c31f1-a7b5-4a58-acb1-3415db68eb06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a15c0b58b74a476da6723e5478415be3'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:32:51.224510 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-fd4c31f1-a7b5-4a58-acb1-3415db68eb06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a15c0b58b74a476da6723e5478415be3'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:32:51.237344 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1299/2598] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:32:51 2022] DELETE /identity/v3/users/a15c0b58b74a476da6723e5478415be3 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:32:51.242058 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290e17b-1d03-4d72-a727-906e6b2c0f78 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:51.242560 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290e17b-1d03-4d72-a727-906e6b2c0f78 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:51.242882 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290e17b-1d03-4d72-a727-906e6b2c0f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:51.262575 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-0290e17b-1d03-4d72-a727-906e6b2c0f78 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:32:51.263804 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-0290e17b-1d03-4d72-a727-906e6b2c0f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:32:51.324254 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1300/2599] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:32:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:32:51.333786 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c31f1-a7b5-4a58-acb1-3415db68eb06 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:32:51.335188 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-fd4c31f1-a7b5-4a58-acb1-3415db68eb06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:32:51.391554 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c31f1-a7b5-4a58-acb1-3415db68eb06 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:32:51.395110 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e732538f-3ae4-420a-b6dc-ddf47e660dfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:32:51.395699 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e732538f-3ae4-420a-b6dc-ddf47e660dfb service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:51.395972 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e732538f-3ae4-420a-b6dc-ddf47e660dfb service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:51.396240 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e732538f-3ae4-420a-b6dc-ddf47e660dfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:51.403311 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:32:51.406046 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e732538f-3ae4-420a-b6dc-ddf47e660dfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:32:51.408333 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e732538f-3ae4-420a-b6dc-ddf47e660dfb service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:32:51.419473 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1300/2600] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:32:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:32:51.586494 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290e17b-1d03-4d72-a727-906e6b2c0f78 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:32:51.588198 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-0290e17b-1d03-4d72-a727-906e6b2c0f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:32:51.600424 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290e17b-1d03-4d72-a727-906e6b2c0f78 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:32:51.605332 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bebd8e-e0ff-486a-9674-9138d15b3a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:32:51.606283 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bebd8e-e0ff-486a-9674-9138d15b3a6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:51.606558 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bebd8e-e0ff-486a-9674-9138d15b3a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:51.606837 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bebd8e-e0ff-486a-9674-9138d15b3a6c admin admin] PATH_INFO: `/v3/projects/119aafe09e994b0fb512e139c894c5e7` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:51.608038 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bebd8e-e0ff-486a-9674-9138d15b3a6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=119aafe09e994b0fb512e139c894c5e7)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:32:51.610432 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bebd8e-e0ff-486a-9674-9138d15b3a6c admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:32:51.659157 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-70bebd8e-e0ff-486a-9674-9138d15b3a6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '119aafe09e994b0fb512e139c894c5e7', 'request_id': 'req-70bebd8e-e0ff-486a-9674-9138d15b3a6c'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:32:51.665119 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-70bebd8e-e0ff-486a-9674-9138d15b3a6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '119aafe09e994b0fb512e139c894c5e7', 'request_id': 'req-70bebd8e-e0ff-486a-9674-9138d15b3a6c'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:32:51.670558 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-70bebd8e-e0ff-486a-9674-9138d15b3a6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '119aafe09e994b0fb512e139c894c5e7', 'request_id': 'req-70bebd8e-e0ff-486a-9674-9138d15b3a6c'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:32:51.682466 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-70bebd8e-e0ff-486a-9674-9138d15b3a6c admin admin] The token cache is being invalidate because project 119aafe09e994b0fb512e139c894c5e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:32:51.682749 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-70bebd8e-e0ff-486a-9674-9138d15b3a6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:32:51.683395 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-70bebd8e-e0ff-486a-9674-9138d15b3a6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:32:51.684843 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1301/2601] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:32:51 2022] DELETE /identity/v3/projects/119aafe09e994b0fb512e139c894c5e7 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:32:51.690610 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a1d87-0443-4a64-9edd-a311efb7f595 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:51.690940 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a1d87-0443-4a64-9edd-a311efb7f595 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:51.691218 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a1d87-0443-4a64-9edd-a311efb7f595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:51.710923 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-ba5a1d87-0443-4a64-9edd-a311efb7f595 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:32:51.712118 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-ba5a1d87-0443-4a64-9edd-a311efb7f595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:32:51.782784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1301/2602] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:32:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:32:51.791682 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079b87f-73bf-4f3c-b406-97dce7de80b3 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:51.791964 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079b87f-73bf-4f3c-b406-97dce7de80b3 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:51.792233 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079b87f-73bf-4f3c-b406-97dce7de80b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:51.814741 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-4079b87f-73bf-4f3c-b406-97dce7de80b3 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:32:51.816372 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-4079b87f-73bf-4f3c-b406-97dce7de80b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:32:51.824100 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1302/2603] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:32:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:32:51.828495 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a1d87-0443-4a64-9edd-a311efb7f595 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:32:51.839004 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a1d87-0443-4a64-9edd-a311efb7f595 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:32:51.843942 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6467883f-40dd-4a46-a64b-31b580cc8783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:32:51.844622 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6467883f-40dd-4a46-a64b-31b580cc8783 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:51.844986 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6467883f-40dd-4a46-a64b-31b580cc8783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:51.845324 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6467883f-40dd-4a46-a64b-31b580cc8783 admin admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:51.846067 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6467883f-40dd-4a46-a64b-31b580cc8783 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:32:51.846478 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6467883f-40dd-4a46-a64b-31b580cc8783 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:32:51.848159 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6467883f-40dd-4a46-a64b-31b580cc8783 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:32:51.860828 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1302/2604] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:32:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:32:51.865014 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4079b87f-73bf-4f3c-b406-97dce7de80b3 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:32:51.874493 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4079b87f-73bf-4f3c-b406-97dce7de80b3 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:32:51.879794 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947864f-5ced-43f6-a91b-7120194c9383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:32:51.880387 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947864f-5ced-43f6-a91b-7120194c9383 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:51.880682 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947864f-5ced-43f6-a91b-7120194c9383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:51.880999 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947864f-5ced-43f6-a91b-7120194c9383 admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:51.881641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b947864f-5ced-43f6-a91b-7120194c9383 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:32:51.883541 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b947864f-5ced-43f6-a91b-7120194c9383 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:32:51.914928 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1303/2605] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:32:51 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:32:51.917891 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6467883f-40dd-4a46-a64b-31b580cc8783 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:32:51.928474 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6467883f-40dd-4a46-a64b-31b580cc8783 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:32:51.985042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e234c4-5939-44ac-b4a3-6ec0c2412356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:32:51.985659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e234c4-5939-44ac-b4a3-6ec0c2412356 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:51.985932 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e234c4-5939-44ac-b4a3-6ec0c2412356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:51.986196 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e234c4-5939-44ac-b4a3-6ec0c2412356 admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:51.986829 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e234c4-5939-44ac-b4a3-6ec0c2412356 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:32:51.988737 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e234c4-5939-44ac-b4a3-6ec0c2412356 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:32:52.011755 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1303/2606] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:32:51 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:32:52.015910 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947864f-5ced-43f6-a91b-7120194c9383 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:32:52.021309 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:32:52.028087 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947864f-5ced-43f6-a91b-7120194c9383 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:32:52.034123 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c4b59-4e4f-430c-a6ee-3acee9a76a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:32:52.035041 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c4b59-4e4f-430c-a6ee-3acee9a76a7f admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:52.035378 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c4b59-4e4f-430c-a6ee-3acee9a76a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:52.035709 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c4b59-4e4f-430c-a6ee-3acee9a76a7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:52.036414 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945c4b59-4e4f-430c-a6ee-3acee9a76a7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:32:52.038132 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945c4b59-4e4f-430c-a6ee-3acee9a76a7f admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:32:52.050484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1304/2607] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:32:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:32:52.054550 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e234c4-5939-44ac-b4a3-6ec0c2412356 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:32:52.065738 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e234c4-5939-44ac-b4a3-6ec0c2412356 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:32:52.070384 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b8cda-553d-4921-9f44-1b38621e9682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:32:52.071146 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b8cda-553d-4921-9f44-1b38621e9682 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:52.071434 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b8cda-553d-4921-9f44-1b38621e9682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:52.071707 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b8cda-553d-4921-9f44-1b38621e9682 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:52.072349 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456b8cda-553d-4921-9f44-1b38621e9682 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:32:52.073945 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456b8cda-553d-4921-9f44-1b38621e9682 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:32:52.084993 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1304/2608] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:32:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:32:52.089910 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c4b59-4e4f-430c-a6ee-3acee9a76a7f admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:32:52.103313 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c4b59-4e4f-430c-a6ee-3acee9a76a7f admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:32:52.108192 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e98e8ca-c2a6-49da-af0d-473ac37659e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:32:52.108882 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98e8ca-c2a6-49da-af0d-473ac37659e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:52.109156 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98e8ca-c2a6-49da-af0d-473ac37659e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:52.109423 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98e8ca-c2a6-49da-af0d-473ac37659e6 admin admin] PATH_INFO: `/v3/projects/71699c60daeb48aaaf5b0611c8f68bc4/users/e4c9bfcd6d1a401194a7176f8aa2e6ac/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:52.126725 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e98e8ca-c2a6-49da-af0d-473ac37659e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71699c60daeb48aaaf5b0611c8f68bc4, user_id=e4c9bfcd6d1a401194a7176f8aa2e6ac, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:32:52.129382 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e98e8ca-c2a6-49da-af0d-473ac37659e6 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:32:52.143685 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1305/2609] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:32:52 2022] PUT /identity/v3/projects/71699c60daeb48aaaf5b0611c8f68bc4/users/e4c9bfcd6d1a401194a7176f8aa2e6ac/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:32:52.149167 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ccf7d0-1dd1-417d-a7f0-189df26b5aca None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:52.149453 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ccf7d0-1dd1-417d-a7f0-189df26b5aca None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:52.149725 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ccf7d0-1dd1-417d-a7f0-189df26b5aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:52.170496 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-f7ccf7d0-1dd1-417d-a7f0-189df26b5aca None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:32:52.171760 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-f7ccf7d0-1dd1-417d-a7f0-189df26b5aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:32:52.239734 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1305/2610] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:32:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:32:52.249680 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e98e8ca-c2a6-49da-af0d-473ac37659e6 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:32:52.251519 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-1e98e8ca-c2a6-49da-af0d-473ac37659e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:32:52.330272 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e98e8ca-c2a6-49da-af0d-473ac37659e6 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:32:52.334662 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd1be8b-e480-465a-b522-51cb0f29ab83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:32:52.335886 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1be8b-e480-465a-b522-51cb0f29ab83 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:52.336445 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1be8b-e480-465a-b522-51cb0f29ab83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:52.336775 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1be8b-e480-465a-b522-51cb0f29ab83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:52.348269 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd1be8b-e480-465a-b522-51cb0f29ab83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:32:52.349824 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd1be8b-e480-465a-b522-51cb0f29ab83 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:32:52.362983 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1306/2611] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:32:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:32:56.640202 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b2770-b90b-4cb9-89e1-ede37bf5914a None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:56.640741 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b2770-b90b-4cb9-89e1-ede37bf5914a None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:56.640842 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b2770-b90b-4cb9-89e1-ede37bf5914a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:56.648797 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:32:56.674617 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-ad6b2770-b90b-4cb9-89e1-ede37bf5914a None None] MFA Rules not processed for user `e4c9bfcd6d1a401194a7176f8aa2e6ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:32:56.675801 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-ad6b2770-b90b-4cb9-89e1-ede37bf5914a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:32:56.774203 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1306/2612] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:32:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:32:56.778721 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd1be8b-e480-465a-b522-51cb0f29ab83 service neutron] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:32:56.789269 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd1be8b-e480-465a-b522-51cb0f29ab83 service neutron] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:32:56.794481 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45640a7-feb3-4692-84f2-a4a4cbd9e622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:32:56.794836 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45640a7-feb3-4692-84f2-a4a4cbd9e622 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:56.795126 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45640a7-feb3-4692-84f2-a4a4cbd9e622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:56.795397 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45640a7-feb3-4692-84f2-a4a4cbd9e622 admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:56.796024 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45640a7-feb3-4692-84f2-a4a4cbd9e622 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:32:56.797955 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45640a7-feb3-4692-84f2-a4a4cbd9e622 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:32:56.839396 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1307/2613] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:32:56 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:32:56.840673 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b2770-b90b-4cb9-89e1-ede37bf5914a None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:32:56.852989 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b2770-b90b-4cb9-89e1-ede37bf5914a None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:32:56.919606 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b41729-54ce-45e1-b9a3-aaec00e75d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:32:56.920220 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b41729-54ce-45e1-b9a3-aaec00e75d7b admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:56.920496 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b41729-54ce-45e1-b9a3-aaec00e75d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:56.920766 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b41729-54ce-45e1-b9a3-aaec00e75d7b admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:56.921411 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b41729-54ce-45e1-b9a3-aaec00e75d7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:32:56.923410 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b41729-54ce-45e1-b9a3-aaec00e75d7b admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:32:56.947833 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1307/2614] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:32:56 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:32:56.951908 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45640a7-feb3-4692-84f2-a4a4cbd9e622 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:32:56.962910 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45640a7-feb3-4692-84f2-a4a4cbd9e622 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:32:56.971324 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04701e-a177-432f-8e28-3294d97b7730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:32:56.972070 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04701e-a177-432f-8e28-3294d97b7730 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:56.972452 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04701e-a177-432f-8e28-3294d97b7730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:56.972852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04701e-a177-432f-8e28-3294d97b7730 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:56.973608 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df04701e-a177-432f-8e28-3294d97b7730 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:32:56.975394 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df04701e-a177-432f-8e28-3294d97b7730 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:32:56.988070 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1308/2615] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:32:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 14:32:56.992614 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b41729-54ce-45e1-b9a3-aaec00e75d7b admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:32:57.007244 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b41729-54ce-45e1-b9a3-aaec00e75d7b admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:32:57.012176 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd4689-298b-41e5-a31e-e7c530da2430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:32:57.012846 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd4689-298b-41e5-a31e-e7c530da2430 admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:57.013124 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd4689-298b-41e5-a31e-e7c530da2430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:57.013399 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd4689-298b-41e5-a31e-e7c530da2430 admin admin] PATH_INFO: `/v3/projects/88e9d7caf24e4ed288bf8b152538daeb/users/b8bbed6164f24411aaa0054004c9d3ad/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:57.033092 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dd4689-298b-41e5-a31e-e7c530da2430 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88e9d7caf24e4ed288bf8b152538daeb, user_id=b8bbed6164f24411aaa0054004c9d3ad, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:32:57.035835 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dd4689-298b-41e5-a31e-e7c530da2430 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:32:57.051711 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1308/2616] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:32:56 2022] PUT /identity/v3/projects/88e9d7caf24e4ed288bf8b152538daeb/users/b8bbed6164f24411aaa0054004c9d3ad/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:32:57.055790 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04701e-a177-432f-8e28-3294d97b7730 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:32:57.067291 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04701e-a177-432f-8e28-3294d97b7730 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:32:57.127977 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca8ba6d-507e-423b-ab09-515c89291079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:32:57.128724 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8ba6d-507e-423b-ab09-515c89291079 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:57.129076 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8ba6d-507e-423b-ab09-515c89291079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:57.129417 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8ba6d-507e-423b-ab09-515c89291079 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:57.130061 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca8ba6d-507e-423b-ab09-515c89291079 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:32:57.131852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca8ba6d-507e-423b-ab09-515c89291079 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:32:57.142400 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1309/2617] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 14:32:57.146714 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd4689-298b-41e5-a31e-e7c530da2430 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:32:57.166396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd4689-298b-41e5-a31e-e7c530da2430 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:32:57.173228 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401df146-1b6c-4b0e-83bf-767b4316473f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:32:57.174490 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401df146-1b6c-4b0e-83bf-767b4316473f admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:32:57.174776 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401df146-1b6c-4b0e-83bf-767b4316473f admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:32:57.175137 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401df146-1b6c-4b0e-83bf-767b4316473f admin admin] PATH_INFO: `/v3/projects/88e9d7caf24e4ed288bf8b152538daeb/users/b8bbed6164f24411aaa0054004c9d3ad/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:32:57.177243 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401df146-1b6c-4b0e-83bf-767b4316473f admin admin] RBAC: Authorizing `identity:create_grant(project_id=88e9d7caf24e4ed288bf8b152538daeb, user_id=b8bbed6164f24411aaa0054004c9d3ad, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:32:57.183657 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401df146-1b6c-4b0e-83bf-767b4316473f admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:32:57.196452 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1309/2618] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:32:57 2022] PUT /identity/v3/projects/88e9d7caf24e4ed288bf8b152538daeb/users/b8bbed6164f24411aaa0054004c9d3ad/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:33:02.351357 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a4d7f-152d-4186-befb-68d45b9c1906 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:33:02.351900 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a4d7f-152d-4186-befb-68d45b9c1906 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:33:02.351900 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a4d7f-152d-4186-befb-68d45b9c1906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:33:02.385430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-617a4d7f-152d-4186-befb-68d45b9c1906 None None] MFA Rules not processed for user `b8bbed6164f24411aaa0054004c9d3ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:33:02.386547 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-617a4d7f-152d-4186-befb-68d45b9c1906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:33:02.414686 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:33:02.490950 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1310/2619] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:33:02 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:33:02.616236 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401df146-1b6c-4b0e-83bf-767b4316473f admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:33:02.618066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-401df146-1b6c-4b0e-83bf-767b4316473f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:33:02.672034 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:33:02.686940 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617a4d7f-152d-4186-befb-68d45b9c1906 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:33:02.706600 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401df146-1b6c-4b0e-83bf-767b4316473f admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:33:02.713391 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceea733-7fb0-4707-b4a2-16a8c7a646e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:33:02.714085 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceea733-7fb0-4707-b4a2-16a8c7a646e1 service nova] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:33:02.714433 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceea733-7fb0-4707-b4a2-16a8c7a646e1 service nova] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:33:02.714778 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceea733-7fb0-4707-b4a2-16a8c7a646e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:33:02.747334 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617a4d7f-152d-4186-befb-68d45b9c1906 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:33:02.752477 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e36feb1-a896-458f-850a-f0251c3cd9c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:33:02.753122 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36feb1-a896-458f-850a-f0251c3cd9c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:33:02.753402 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36feb1-a896-458f-850a-f0251c3cd9c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:33:02.753693 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36feb1-a896-458f-850a-f0251c3cd9c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:33:02.755659 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-1e36feb1-a896-458f-850a-f0251c3cd9c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:33:02.778382 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceea733-7fb0-4707-b4a2-16a8c7a646e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:33:02.780289 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceea733-7fb0-4707-b4a2-16a8c7a646e1 service nova] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:33:02.790879 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1310/2620] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:33:02 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:33:02.848192 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e36feb1-a896-458f-850a-f0251c3cd9c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:33:02.848192 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e36feb1-a896-458f-850a-f0251c3cd9c5 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:33:02.872715 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1311/2621] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:33:02 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 14:33:03.114068 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66e4c0-e6bf-4b8c-bbbb-03cac1be7aae None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:33:03.114368 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66e4c0-e6bf-4b8c-bbbb-03cac1be7aae None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:33:03.114648 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66e4c0-e6bf-4b8c-bbbb-03cac1be7aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:33:03.146150 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-db66e4c0-e6bf-4b8c-bbbb-03cac1be7aae None None] MFA Rules not processed for user `82328b7fd50649a485c6eaf27c5597c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:33:03.147572 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-db66e4c0-e6bf-4b8c-bbbb-03cac1be7aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:33:03.250562 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1311/2622] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 14:33:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:33:03.282042 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e36feb1-a896-458f-850a-f0251c3cd9c5 service neutron] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:33:03.283992 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-1e36feb1-a896-458f-850a-f0251c3cd9c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:33:03.385040 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e36feb1-a896-458f-850a-f0251c3cd9c5 service neutron] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:33:03.390283 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5632834b-212c-4873-945e-530990678b67 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '879f296733114f6f844a98aaefe35e92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:33:03.391166 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632834b-212c-4873-945e-530990678b67 service swift] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:33:03.391575 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632834b-212c-4873-945e-530990678b67 service swift] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:33:03.391910 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632834b-212c-4873-945e-530990678b67 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:33:03.402511 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5632834b-212c-4873-945e-530990678b67 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:33:03.408025 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5632834b-212c-4873-945e-530990678b67 service swift] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:33:03.409161 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:33:03.419441 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1312/2623] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 14:33:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 24 14:33:04.550451 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72700262-23b6-4ce3-9f71-d7c6a97f06ff None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:33:04.550990 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72700262-23b6-4ce3-9f71-d7c6a97f06ff None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:33:04.551103 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72700262-23b6-4ce3-9f71-d7c6a97f06ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:33:04.573827 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-72700262-23b6-4ce3-9f71-d7c6a97f06ff None None] MFA Rules not processed for user `e4c9bfcd6d1a401194a7176f8aa2e6ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:33:04.575190 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-72700262-23b6-4ce3-9f71-d7c6a97f06ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:33:04.585332 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1312/2624] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:33:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:33:05.565441 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5632834b-212c-4873-945e-530990678b67 service swift] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:33:05.567455 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-5632834b-212c-4873-945e-530990678b67 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:33:05.588235 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5632834b-212c-4873-945e-530990678b67 service swift] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:33:05.594920 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eebaaf-ef5d-4e73-a8c6-efb1552024b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:33:05.595589 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eebaaf-ef5d-4e73-a8c6-efb1552024b8 service nova] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:33:05.595875 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eebaaf-ef5d-4e73-a8c6-efb1552024b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:33:05.596159 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eebaaf-ef5d-4e73-a8c6-efb1552024b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:33:05.598670 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-75eebaaf-ef5d-4e73-a8c6-efb1552024b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:33:05.615723 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eebaaf-ef5d-4e73-a8c6-efb1552024b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:33:05.617487 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eebaaf-ef5d-4e73-a8c6-efb1552024b8 service nova] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:33:05.638981 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1313/2625] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:33:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:33:11.324236 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc26ec-1572-44a2-a7fd-2c269c65e874 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:33:11.324750 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc26ec-1572-44a2-a7fd-2c269c65e874 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:33:11.324750 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc26ec-1572-44a2-a7fd-2c269c65e874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:33:11.344505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-51dc26ec-1572-44a2-a7fd-2c269c65e874 None None] MFA Rules not processed for user `c3cf8e31062d4bd7bfa76c09e7e2adea`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:33:11.345695 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-51dc26ec-1572-44a2-a7fd-2c269c65e874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:33:11.403755 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1313/2626] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:33:11 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:33:15.627165 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=7 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:33:21.395381 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=89 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:33:29.859664 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eebaaf-ef5d-4e73-a8c6-efb1552024b8 service nova] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:33:29.868851 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eebaaf-ef5d-4e73-a8c6-efb1552024b8 service nova] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:33:29.872388 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37d9f1-df90-4eba-8b00-bfe69000caf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:33:29.872977 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37d9f1-df90-4eba-8b00-bfe69000caf1 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:33:29.873252 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37d9f1-df90-4eba-8b00-bfe69000caf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:33:29.873531 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37d9f1-df90-4eba-8b00-bfe69000caf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:33:29.875786 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-cc37d9f1-df90-4eba-8b00-bfe69000caf1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:33:29.884922 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc37d9f1-df90-4eba-8b00-bfe69000caf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:33:29.886325 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc37d9f1-df90-4eba-8b00-bfe69000caf1 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:33:29.894597 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1314/2627] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:33:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:33:39.890615 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=6 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:33:48.223012 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc26ec-1572-44a2-a7fd-2c269c65e874 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:33:48.224416 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-51dc26ec-1572-44a2-a7fd-2c269c65e874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:33:48.236483 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc26ec-1572-44a2-a7fd-2c269c65e874 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:33:48.239985 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba98d5-6046-43e8-868d-4e843eb09617 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:33:48.240573 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba98d5-6046-43e8-868d-4e843eb09617 service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:33:48.240845 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba98d5-6046-43e8-868d-4e843eb09617 service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:33:48.241110 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba98d5-6046-43e8-868d-4e843eb09617 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:33:48.250482 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ba98d5-6046-43e8-868d-4e843eb09617 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:33:48.251947 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ba98d5-6046-43e8-868d-4e843eb09617 service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:33:48.261578 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1314/2628] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:33:48 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:33:58.257090 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=6 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:34:28.251688 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37d9f1-df90-4eba-8b00-bfe69000caf1 service neutron] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:34:28.253093 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-cc37d9f1-df90-4eba-8b00-bfe69000caf1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:34:28.411943 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37d9f1-df90-4eba-8b00-bfe69000caf1 service neutron] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:34:28.417406 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a159c38-9ab6-4dec-91b6-d0b5b783be79 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18fe384ecd454fb298dcc788f41dd44b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:34:28.418018 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a159c38-9ab6-4dec-91b6-d0b5b783be79 service placement] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:34:28.418296 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a159c38-9ab6-4dec-91b6-d0b5b783be79 service placement] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:34:28.418616 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a159c38-9ab6-4dec-91b6-d0b5b783be79 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:34:28.420443 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-5a159c38-9ab6-4dec-91b6-d0b5b783be79 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:34:28.434348 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a159c38-9ab6-4dec-91b6-d0b5b783be79 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:34:28.436528 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a159c38-9ab6-4dec-91b6-d0b5b783be79 service placement] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:34:28.447622 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1315/2629] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:34:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:34:32.132219 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aed5b8-ef57-4fab-8caf-b7a6ee5b6e4f None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:34:32.132711 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aed5b8-ef57-4fab-8caf-b7a6ee5b6e4f None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:34:32.132711 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aed5b8-ef57-4fab-8caf-b7a6ee5b6e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:34:32.151423 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-d2aed5b8-ef57-4fab-8caf-b7a6ee5b6e4f None None] MFA Rules not processed for user `883e6570afb84a099e46cdace5a88b2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:34:32.152614 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-d2aed5b8-ef57-4fab-8caf-b7a6ee5b6e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:34:32.207986 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1315/2630] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:34:32 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:34:38.442885 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=63 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:34:42.200702 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=25 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:34:55.140851 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a159c38-9ab6-4dec-91b6-d0b5b783be79 service placement] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:34:55.159802 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a159c38-9ab6-4dec-91b6-d0b5b783be79 service placement] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:34:55.163340 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd788380-37ea-4182-abb8-80aa1599ede5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:34:55.163933 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd788380-37ea-4182-abb8-80aa1599ede5 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:34:55.164205 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd788380-37ea-4182-abb8-80aa1599ede5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:34:55.164475 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd788380-37ea-4182-abb8-80aa1599ede5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:34:55.166201 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-dd788380-37ea-4182-abb8-80aa1599ede5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:34:55.248353 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd788380-37ea-4182-abb8-80aa1599ede5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:34:55.249790 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd788380-37ea-4182-abb8-80aa1599ede5 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:34:55.268196 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1316/2631] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:34:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:34:58.515336 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aed5b8-ef57-4fab-8caf-b7a6ee5b6e4f None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:34:58.516738 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-d2aed5b8-ef57-4fab-8caf-b7a6ee5b6e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:34:58.533763 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aed5b8-ef57-4fab-8caf-b7a6ee5b6e4f None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:34:58.539420 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36248319-05ec-4051-ba4d-b8ce13575def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:34:58.540132 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36248319-05ec-4051-ba4d-b8ce13575def admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:34:58.540477 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36248319-05ec-4051-ba4d-b8ce13575def admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:34:58.540828 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36248319-05ec-4051-ba4d-b8ce13575def admin admin] PATH_INFO: `/v3/users/c3cf8e31062d4bd7bfa76c09e7e2adea` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:34:58.542120 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36248319-05ec-4051-ba4d-b8ce13575def admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3cf8e31062d4bd7bfa76c09e7e2adea)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:34:58.544273 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36248319-05ec-4051-ba4d-b8ce13575def admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:34:58.593089 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-36248319-05ec-4051-ba4d-b8ce13575def admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3cf8e31062d4bd7bfa76c09e7e2adea'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:34:58.593738 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-36248319-05ec-4051-ba4d-b8ce13575def admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3cf8e31062d4bd7bfa76c09e7e2adea'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:34:58.600664 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-36248319-05ec-4051-ba4d-b8ce13575def admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3cf8e31062d4bd7bfa76c09e7e2adea'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:34:58.601289 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-36248319-05ec-4051-ba4d-b8ce13575def admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3cf8e31062d4bd7bfa76c09e7e2adea'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:34:58.613178 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-36248319-05ec-4051-ba4d-b8ce13575def admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3cf8e31062d4bd7bfa76c09e7e2adea'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:34:58.625845 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1316/2632] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:34:58 2022] DELETE /identity/v3/users/c3cf8e31062d4bd7bfa76c09e7e2adea => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:34:58.798961 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd788380-37ea-4182-abb8-80aa1599ede5 service neutron] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:34:58.809031 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd788380-37ea-4182-abb8-80aa1599ede5 service neutron] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:34:58.872389 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579738a-9945-48b1-8bdb-2466a4d6a293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:34:58.873014 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579738a-9945-48b1-8bdb-2466a4d6a293 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:34:58.873297 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579738a-9945-48b1-8bdb-2466a4d6a293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:34:58.874077 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579738a-9945-48b1-8bdb-2466a4d6a293 admin admin] PATH_INFO: `/v3/projects/9b458789f89a441caad76e78167bc00b` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:34:58.875234 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9579738a-9945-48b1-8bdb-2466a4d6a293 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b458789f89a441caad76e78167bc00b)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:34:58.877120 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9579738a-9945-48b1-8bdb-2466a4d6a293 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:34:58.922878 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9579738a-9945-48b1-8bdb-2466a4d6a293 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b458789f89a441caad76e78167bc00b', 'request_id': 'req-9579738a-9945-48b1-8bdb-2466a4d6a293'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:34:58.928858 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9579738a-9945-48b1-8bdb-2466a4d6a293 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b458789f89a441caad76e78167bc00b', 'request_id': 'req-9579738a-9945-48b1-8bdb-2466a4d6a293'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:34:58.932704 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:34:58.935113 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9579738a-9945-48b1-8bdb-2466a4d6a293 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b458789f89a441caad76e78167bc00b', 'request_id': 'req-9579738a-9945-48b1-8bdb-2466a4d6a293'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:34:58.948161 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9579738a-9945-48b1-8bdb-2466a4d6a293 admin admin] The token cache is being invalidate because project 9b458789f89a441caad76e78167bc00b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:34:58.948467 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9579738a-9945-48b1-8bdb-2466a4d6a293 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:34:58.949066 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9579738a-9945-48b1-8bdb-2466a4d6a293 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:34:58.950514 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1317/2633] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:34:58 2022] DELETE /identity/v3/projects/9b458789f89a441caad76e78167bc00b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:34:58.956228 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96993b5f-905a-476f-95c3-b10d6a6050fa None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:34:58.956511 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96993b5f-905a-476f-95c3-b10d6a6050fa None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:34:58.956784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96993b5f-905a-476f-95c3-b10d6a6050fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:34:59.011754 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-96993b5f-905a-476f-95c3-b10d6a6050fa None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:34:59.012868 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-96993b5f-905a-476f-95c3-b10d6a6050fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:34:59.070550 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1317/2634] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:34:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:34:59.080285 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb118b26-ee1f-4d93-9a76-1b1b6f359cd4 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:34:59.080574 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb118b26-ee1f-4d93-9a76-1b1b6f359cd4 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:34:59.080847 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb118b26-ee1f-4d93-9a76-1b1b6f359cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:34:59.100533 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-bb118b26-ee1f-4d93-9a76-1b1b6f359cd4 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:34:59.102054 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-bb118b26-ee1f-4d93-9a76-1b1b6f359cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:34:59.109862 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1318/2635] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:34:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:34:59.114098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96993b5f-905a-476f-95c3-b10d6a6050fa None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:34:59.124122 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96993b5f-905a-476f-95c3-b10d6a6050fa None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:34:59.128600 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed5cbd-6a56-441d-aa67-24dc797aa761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:34:59.129191 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed5cbd-6a56-441d-aa67-24dc797aa761 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:34:59.129467 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed5cbd-6a56-441d-aa67-24dc797aa761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:34:59.129732 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed5cbd-6a56-441d-aa67-24dc797aa761 admin admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:34:59.130371 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaed5cbd-6a56-441d-aa67-24dc797aa761 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:34:59.130669 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaed5cbd-6a56-441d-aa67-24dc797aa761 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:34:59.132285 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaed5cbd-6a56-441d-aa67-24dc797aa761 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:34:59.143998 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1318/2636] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:34:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:34:59.148106 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb118b26-ee1f-4d93-9a76-1b1b6f359cd4 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:34:59.157165 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb118b26-ee1f-4d93-9a76-1b1b6f359cd4 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:34:59.161662 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395f110-bf7b-492f-8548-96b470061a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:34:59.162244 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395f110-bf7b-492f-8548-96b470061a2e admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:34:59.162516 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395f110-bf7b-492f-8548-96b470061a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:34:59.162782 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395f110-bf7b-492f-8548-96b470061a2e admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:34:59.163432 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7395f110-bf7b-492f-8548-96b470061a2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:34:59.165345 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7395f110-bf7b-492f-8548-96b470061a2e admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:34:59.196118 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1319/2637] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:34:59 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:34:59.197409 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed5cbd-6a56-441d-aa67-24dc797aa761 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:34:59.207617 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed5cbd-6a56-441d-aa67-24dc797aa761 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:34:59.232608 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:34:59.263852 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb801fbc-681f-4b3e-bd40-98253a55337f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:34:59.264902 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb801fbc-681f-4b3e-bd40-98253a55337f admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:34:59.265179 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb801fbc-681f-4b3e-bd40-98253a55337f admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:34:59.265444 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb801fbc-681f-4b3e-bd40-98253a55337f admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:34:59.266066 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb801fbc-681f-4b3e-bd40-98253a55337f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:34:59.268017 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb801fbc-681f-4b3e-bd40-98253a55337f admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:34:59.289448 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1319/2638] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:34:59 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:34:59.293545 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395f110-bf7b-492f-8548-96b470061a2e admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:34:59.303669 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395f110-bf7b-492f-8548-96b470061a2e admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:34:59.308168 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44da74b-6347-4a0e-a719-07432b25779f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:34:59.308762 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44da74b-6347-4a0e-a719-07432b25779f admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:34:59.309036 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44da74b-6347-4a0e-a719-07432b25779f admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:34:59.309305 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44da74b-6347-4a0e-a719-07432b25779f admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:34:59.310051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44da74b-6347-4a0e-a719-07432b25779f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:34:59.311694 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44da74b-6347-4a0e-a719-07432b25779f admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:34:59.321387 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1320/2639] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:34:59.325160 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb801fbc-681f-4b3e-bd40-98253a55337f admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:34:59.335542 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb801fbc-681f-4b3e-bd40-98253a55337f admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:34:59.340225 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3359015-1f4d-43ad-867b-78cfdf632313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:34:59.340819 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3359015-1f4d-43ad-867b-78cfdf632313 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:34:59.341094 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3359015-1f4d-43ad-867b-78cfdf632313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:34:59.341360 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3359015-1f4d-43ad-867b-78cfdf632313 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:34:59.341995 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3359015-1f4d-43ad-867b-78cfdf632313 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:34:59.344173 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3359015-1f4d-43ad-867b-78cfdf632313 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:34:59.355919 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1320/2640] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:34:59.360047 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44da74b-6347-4a0e-a719-07432b25779f admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:34:59.369903 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44da74b-6347-4a0e-a719-07432b25779f admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:34:59.374652 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deae4b7-68ac-41f5-a724-3c26f11267ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:34:59.375339 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deae4b7-68ac-41f5-a724-3c26f11267ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:34:59.375616 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deae4b7-68ac-41f5-a724-3c26f11267ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:34:59.375888 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deae4b7-68ac-41f5-a724-3c26f11267ce admin admin] PATH_INFO: `/v3/projects/21710ac430424ec9bf33aabfda5eb8ba/users/abb529f39710407abf9d6e1cedd9a603/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:34:59.390639 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deae4b7-68ac-41f5-a724-3c26f11267ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=21710ac430424ec9bf33aabfda5eb8ba, user_id=abb529f39710407abf9d6e1cedd9a603, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:34:59.393516 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deae4b7-68ac-41f5-a724-3c26f11267ce admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:34:59.406842 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1321/2641] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:34:59 2022] PUT /identity/v3/projects/21710ac430424ec9bf33aabfda5eb8ba/users/abb529f39710407abf9d6e1cedd9a603/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:34:59.412006 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9bb0b6-0ea5-4a30-9ad7-7a1a1c19d2fc None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:34:59.412289 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9bb0b6-0ea5-4a30-9ad7-7a1a1c19d2fc None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:34:59.412560 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9bb0b6-0ea5-4a30-9ad7-7a1a1c19d2fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:34:59.430893 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-ff9bb0b6-0ea5-4a30-9ad7-7a1a1c19d2fc None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:34:59.432074 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-ff9bb0b6-0ea5-4a30-9ad7-7a1a1c19d2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:34:59.488604 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1321/2642] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:34:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:34:59.498274 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deae4b7-68ac-41f5-a724-3c26f11267ce admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:34:59.500005 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-2deae4b7-68ac-41f5-a724-3c26f11267ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:34:59.566587 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deae4b7-68ac-41f5-a724-3c26f11267ce admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:34:59.570149 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e29aa0-19dd-401b-a334-26dc109f2b54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:34:59.570741 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e29aa0-19dd-401b-a334-26dc109f2b54 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:34:59.571043 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e29aa0-19dd-401b-a334-26dc109f2b54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:34:59.571312 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e29aa0-19dd-401b-a334-26dc109f2b54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:34:59.580933 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e29aa0-19dd-401b-a334-26dc109f2b54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:34:59.582372 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e29aa0-19dd-401b-a334-26dc109f2b54 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:34:59.591609 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1322/2643] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:34:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 14:35:03.468794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c03503-4fca-45d8-83dd-46dbb24fffee None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:35:03.469286 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c03503-4fca-45d8-83dd-46dbb24fffee None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:35:03.469383 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c03503-4fca-45d8-83dd-46dbb24fffee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:35:03.497543 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-72c03503-4fca-45d8-83dd-46dbb24fffee None None] MFA Rules not processed for user `abb529f39710407abf9d6e1cedd9a603`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:35:03.498702 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-72c03503-4fca-45d8-83dd-46dbb24fffee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:35:03.564790 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1322/2644] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:35:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:35:03.580101 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e29aa0-19dd-401b-a334-26dc109f2b54 service neutron] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:35:03.581568 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-66e29aa0-19dd-401b-a334-26dc109f2b54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:35:03.667861 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:35:03.702000 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e29aa0-19dd-401b-a334-26dc109f2b54 service neutron] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:35:03.707531 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0463a58e-52af-4876-b2ff-40443cca3b82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:35:03.708129 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463a58e-52af-4876-b2ff-40443cca3b82 service nova] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:35:03.708404 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463a58e-52af-4876-b2ff-40443cca3b82 service nova] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:35:03.708678 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463a58e-52af-4876-b2ff-40443cca3b82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:35:03.716507 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0463a58e-52af-4876-b2ff-40443cca3b82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:35:03.718128 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0463a58e-52af-4876-b2ff-40443cca3b82 service nova] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:35:03.727596 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1323/2645] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:35:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:35:05.300402 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92eff4-95de-4ac7-a4ee-78eb4c4d5fed None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:35:05.300878 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92eff4-95de-4ac7-a4ee-78eb4c4d5fed None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:35:05.301410 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92eff4-95de-4ac7-a4ee-78eb4c4d5fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:35:05.323528 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-cc92eff4-95de-4ac7-a4ee-78eb4c4d5fed None None] MFA Rules not processed for user `abb529f39710407abf9d6e1cedd9a603`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:35:05.324750 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-cc92eff4-95de-4ac7-a4ee-78eb4c4d5fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:35:05.331937 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1323/2646] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:35:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:35:13.723196 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=12 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:35:15.316418 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=83,keystone:INSERT=3 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:36:05.884497 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900ae64-3530-4da8-bf0b-6e639200960d None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:36:05.884962 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900ae64-3530-4da8-bf0b-6e639200960d None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:36:05.885057 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900ae64-3530-4da8-bf0b-6e639200960d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:36:05.914663 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-3900ae64-3530-4da8-bf0b-6e639200960d None None] MFA Rules not processed for user `82328b7fd50649a485c6eaf27c5597c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:36:05.915871 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-3900ae64-3530-4da8-bf0b-6e639200960d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:36:05.969616 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1324/2647] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 14:36:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:36:06.014700 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc92eff4-95de-4ac7-a4ee-78eb4c4d5fed None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:36:06.016126 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-cc92eff4-95de-4ac7-a4ee-78eb4c4d5fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:36:06.083121 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc92eff4-95de-4ac7-a4ee-78eb4c4d5fed None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:36:06.087972 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23f99b-51a9-476e-b179-dbbdd39739cc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '879f296733114f6f844a98aaefe35e92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:36:06.088565 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23f99b-51a9-476e-b179-dbbdd39739cc service swift] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:36:06.088837 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23f99b-51a9-476e-b179-dbbdd39739cc service swift] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:36:06.089107 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23f99b-51a9-476e-b179-dbbdd39739cc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:36:06.097409 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd23f99b-51a9-476e-b179-dbbdd39739cc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:36:06.099001 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd23f99b-51a9-476e-b179-dbbdd39739cc service swift] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:36:06.106920 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1324/2648] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 14:36:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 14:36:12.606785 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900ae64-3530-4da8-bf0b-6e639200960d None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:36:12.608341 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-3900ae64-3530-4da8-bf0b-6e639200960d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:36:12.624392 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900ae64-3530-4da8-bf0b-6e639200960d None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:36:12.629091 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a1876-7d56-41da-a1ae-4879b2288900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:36:12.629832 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a1876-7d56-41da-a1ae-4879b2288900 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:36:12.630170 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a1876-7d56-41da-a1ae-4879b2288900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:36:12.630658 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a1876-7d56-41da-a1ae-4879b2288900 admin admin] PATH_INFO: `/v3/users/27724ba162164bbb8faae54388ff11d4` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:36:12.631921 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a1876-7d56-41da-a1ae-4879b2288900 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27724ba162164bbb8faae54388ff11d4)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:36:12.633911 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a1876-7d56-41da-a1ae-4879b2288900 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:36:12.688162 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-fa6a1876-7d56-41da-a1ae-4879b2288900 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27724ba162164bbb8faae54388ff11d4'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:36:12.688817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-fa6a1876-7d56-41da-a1ae-4879b2288900 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27724ba162164bbb8faae54388ff11d4'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:36:12.689439 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-fa6a1876-7d56-41da-a1ae-4879b2288900 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27724ba162164bbb8faae54388ff11d4'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:36:12.701652 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-fa6a1876-7d56-41da-a1ae-4879b2288900 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27724ba162164bbb8faae54388ff11d4'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:36:12.708552 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-fa6a1876-7d56-41da-a1ae-4879b2288900 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27724ba162164bbb8faae54388ff11d4'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:36:12.722414 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1325/2649] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:36:12 2022] DELETE /identity/v3/users/27724ba162164bbb8faae54388ff11d4 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:36:12.894039 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23f99b-51a9-476e-b179-dbbdd39739cc service swift] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:36:12.905182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23f99b-51a9-476e-b179-dbbdd39739cc service swift] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:36:12.961581 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc7321-8ed4-41e8-ba9f-326f78dde442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:36:12.962205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc7321-8ed4-41e8-ba9f-326f78dde442 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:36:12.962483 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc7321-8ed4-41e8-ba9f-326f78dde442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:36:12.962753 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc7321-8ed4-41e8-ba9f-326f78dde442 admin admin] PATH_INFO: `/v3/projects/8830800be89540929716703bdbb07b13` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:36:12.963893 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adc7321-8ed4-41e8-ba9f-326f78dde442 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8830800be89540929716703bdbb07b13)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:36:12.965824 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adc7321-8ed4-41e8-ba9f-326f78dde442 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:36:13.011029 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-4adc7321-8ed4-41e8-ba9f-326f78dde442 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8830800be89540929716703bdbb07b13', 'request_id': 'req-4adc7321-8ed4-41e8-ba9f-326f78dde442'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:36:13.017266 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-4adc7321-8ed4-41e8-ba9f-326f78dde442 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8830800be89540929716703bdbb07b13', 'request_id': 'req-4adc7321-8ed4-41e8-ba9f-326f78dde442'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:36:13.023010 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-4adc7321-8ed4-41e8-ba9f-326f78dde442 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8830800be89540929716703bdbb07b13', 'request_id': 'req-4adc7321-8ed4-41e8-ba9f-326f78dde442'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:36:13.038094 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-4adc7321-8ed4-41e8-ba9f-326f78dde442 admin admin] The token cache is being invalidate because project 8830800be89540929716703bdbb07b13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:36:13.038388 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-4adc7321-8ed4-41e8-ba9f-326f78dde442 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:36:13.039076 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-4adc7321-8ed4-41e8-ba9f-326f78dde442 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:36:13.040612 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1325/2650] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:36:12 2022] DELETE /identity/v3/projects/8830800be89540929716703bdbb07b13 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:36:13.045873 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a1876-7d56-41da-a1ae-4879b2288900 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:36:13.048051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-fa6a1876-7d56-41da-a1ae-4879b2288900 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:36:13.063545 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a1876-7d56-41da-a1ae-4879b2288900 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:36:13.119153 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb41c842-364a-4110-8143-bcccbd50364a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:36:13.119780 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41c842-364a-4110-8143-bcccbd50364a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:36:13.120092 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41c842-364a-4110-8143-bcccbd50364a admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:36:13.120365 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41c842-364a-4110-8143-bcccbd50364a admin admin] PATH_INFO: `/v3/users/4bc2c1ce1de14353b6aa6e02785dfebd` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:36:13.121494 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb41c842-364a-4110-8143-bcccbd50364a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bc2c1ce1de14353b6aa6e02785dfebd)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:36:13.123478 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb41c842-364a-4110-8143-bcccbd50364a admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:36:13.133264 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:36:13.169962 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-bb41c842-364a-4110-8143-bcccbd50364a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bc2c1ce1de14353b6aa6e02785dfebd'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:36:13.170544 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-bb41c842-364a-4110-8143-bcccbd50364a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bc2c1ce1de14353b6aa6e02785dfebd'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:36:13.171164 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-bb41c842-364a-4110-8143-bcccbd50364a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bc2c1ce1de14353b6aa6e02785dfebd'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:36:13.182781 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-bb41c842-364a-4110-8143-bcccbd50364a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bc2c1ce1de14353b6aa6e02785dfebd'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:36:13.189734 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-bb41c842-364a-4110-8143-bcccbd50364a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bc2c1ce1de14353b6aa6e02785dfebd'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:36:13.201653 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1326/2651] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:36:13 2022] DELETE /identity/v3/users/4bc2c1ce1de14353b6aa6e02785dfebd => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:36:13.368488 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc7321-8ed4-41e8-ba9f-326f78dde442 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:36:13.369951 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-4adc7321-8ed4-41e8-ba9f-326f78dde442 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:36:13.383104 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc7321-8ed4-41e8-ba9f-326f78dde442 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:36:13.401160 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:36:13.444009 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e717b14-8596-4e66-9c67-5c4c772ac568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:36:13.444551 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e717b14-8596-4e66-9c67-5c4c772ac568 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:36:13.444825 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e717b14-8596-4e66-9c67-5c4c772ac568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:36:13.445091 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e717b14-8596-4e66-9c67-5c4c772ac568 admin admin] PATH_INFO: `/v3/projects/e314f9937c1f46d88e3784e70f63a952` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:36:13.446192 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e717b14-8596-4e66-9c67-5c4c772ac568 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e314f9937c1f46d88e3784e70f63a952)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:36:13.448117 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e717b14-8596-4e66-9c67-5c4c772ac568 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:36:13.494453 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-1e717b14-8596-4e66-9c67-5c4c772ac568 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e314f9937c1f46d88e3784e70f63a952', 'request_id': 'req-1e717b14-8596-4e66-9c67-5c4c772ac568'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:36:13.500491 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-1e717b14-8596-4e66-9c67-5c4c772ac568 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e314f9937c1f46d88e3784e70f63a952', 'request_id': 'req-1e717b14-8596-4e66-9c67-5c4c772ac568'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:36:13.505975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-1e717b14-8596-4e66-9c67-5c4c772ac568 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e314f9937c1f46d88e3784e70f63a952', 'request_id': 'req-1e717b14-8596-4e66-9c67-5c4c772ac568'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:36:13.518178 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-1e717b14-8596-4e66-9c67-5c4c772ac568 admin admin] The token cache is being invalidate because project e314f9937c1f46d88e3784e70f63a952 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:36:13.518463 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-1e717b14-8596-4e66-9c67-5c4c772ac568 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:36:13.519109 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-1e717b14-8596-4e66-9c67-5c4c772ac568 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:36:13.520558 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1326/2652] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:36:13 2022] DELETE /identity/v3/projects/e314f9937c1f46d88e3784e70f63a952 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:36:13.525535 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9afa08b-dbdd-469f-a36e-0f69873a3782 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:36:13.525822 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9afa08b-dbdd-469f-a36e-0f69873a3782 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:36:13.526097 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9afa08b-dbdd-469f-a36e-0f69873a3782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:36:13.546130 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-f9afa08b-dbdd-469f-a36e-0f69873a3782 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:36:13.547383 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-f9afa08b-dbdd-469f-a36e-0f69873a3782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:36:13.605510 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1327/2653] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:36:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:36:13.613133 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004aec9b-77d6-4fff-b941-0b002f6a94e7 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:36:13.613415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004aec9b-77d6-4fff-b941-0b002f6a94e7 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:36:13.613687 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004aec9b-77d6-4fff-b941-0b002f6a94e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:36:13.633165 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-004aec9b-77d6-4fff-b941-0b002f6a94e7 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:36:13.634361 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-004aec9b-77d6-4fff-b941-0b002f6a94e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:36:13.642794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1327/2654] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:36:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:36:13.646979 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9afa08b-dbdd-469f-a36e-0f69873a3782 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:36:13.657343 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9afa08b-dbdd-469f-a36e-0f69873a3782 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:36:13.661860 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e3bc3-5e2d-44d3-a850-3d71574827b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:36:13.662442 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e3bc3-5e2d-44d3-a850-3d71574827b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:36:13.662720 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e3bc3-5e2d-44d3-a850-3d71574827b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:36:13.663021 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e3bc3-5e2d-44d3-a850-3d71574827b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:36:13.663656 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e3bc3-5e2d-44d3-a850-3d71574827b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:36:13.663954 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e3bc3-5e2d-44d3-a850-3d71574827b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:36:13.665531 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e3bc3-5e2d-44d3-a850-3d71574827b8 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:36:13.678075 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1328/2655] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:36:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:36:13.682245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004aec9b-77d6-4fff-b941-0b002f6a94e7 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:36:13.691597 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004aec9b-77d6-4fff-b941-0b002f6a94e7 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:36:13.698050 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54fce9d-2690-49fb-8a72-23acbc6fbdc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:36:13.698634 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54fce9d-2690-49fb-8a72-23acbc6fbdc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:36:13.698951 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54fce9d-2690-49fb-8a72-23acbc6fbdc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:36:13.699247 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54fce9d-2690-49fb-8a72-23acbc6fbdc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:36:13.700426 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54fce9d-2690-49fb-8a72-23acbc6fbdc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:36:13.702428 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54fce9d-2690-49fb-8a72-23acbc6fbdc2 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:36:13.743004 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1328/2656] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:36:13 2022] POST /identity/v3/projects => generated 379 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:36:13.747661 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e3bc3-5e2d-44d3-a850-3d71574827b8 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:36:13.757922 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e3bc3-5e2d-44d3-a850-3d71574827b8 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:36:13.813369 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab49325b-6b66-417a-a2ec-9935b92a35bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:36:13.813966 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab49325b-6b66-417a-a2ec-9935b92a35bc admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:36:13.814252 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab49325b-6b66-417a-a2ec-9935b92a35bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:36:13.814536 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab49325b-6b66-417a-a2ec-9935b92a35bc admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:36:13.815289 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab49325b-6b66-417a-a2ec-9935b92a35bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:36:13.817123 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab49325b-6b66-417a-a2ec-9935b92a35bc admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:36:13.839303 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1329/2657] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:36:13 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:36:13.843388 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54fce9d-2690-49fb-8a72-23acbc6fbdc2 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:36:13.853717 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54fce9d-2690-49fb-8a72-23acbc6fbdc2 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:36:13.858449 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7ca28c-87b2-45bd-98eb-0a7652428fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:36:13.859083 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ca28c-87b2-45bd-98eb-0a7652428fda admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:36:13.859363 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ca28c-87b2-45bd-98eb-0a7652428fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:36:13.859635 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ca28c-87b2-45bd-98eb-0a7652428fda admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:36:13.860278 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7ca28c-87b2-45bd-98eb-0a7652428fda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:36:13.861859 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7ca28c-87b2-45bd-98eb-0a7652428fda admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:36:13.871665 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1329/2658] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:36:13.875904 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab49325b-6b66-417a-a2ec-9935b92a35bc admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:36:13.882210 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:36:13.886003 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab49325b-6b66-417a-a2ec-9935b92a35bc admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:36:13.892781 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b097f394-08aa-498d-ae67-6a6b6fd296d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:36:13.893572 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097f394-08aa-498d-ae67-6a6b6fd296d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:36:13.894026 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097f394-08aa-498d-ae67-6a6b6fd296d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:36:13.894373 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097f394-08aa-498d-ae67-6a6b6fd296d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:36:13.895299 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b097f394-08aa-498d-ae67-6a6b6fd296d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:36:13.896887 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b097f394-08aa-498d-ae67-6a6b6fd296d3 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:36:13.907824 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1330/2659] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:36:13.912151 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7ca28c-87b2-45bd-98eb-0a7652428fda admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:36:13.922062 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7ca28c-87b2-45bd-98eb-0a7652428fda admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:36:13.926548 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac2fee2-0e27-4b24-9bb2-360e2d851e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:36:13.927219 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac2fee2-0e27-4b24-9bb2-360e2d851e1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:36:13.927495 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac2fee2-0e27-4b24-9bb2-360e2d851e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:36:13.927763 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac2fee2-0e27-4b24-9bb2-360e2d851e1c admin admin] PATH_INFO: `/v3/projects/f69d85fa8dc545efb55a0b2949e9f06c/users/ffe9a464991e4965a0638d0c23b17ae7/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:36:13.942684 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac2fee2-0e27-4b24-9bb2-360e2d851e1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f69d85fa8dc545efb55a0b2949e9f06c, user_id=ffe9a464991e4965a0638d0c23b17ae7, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:36:13.945343 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac2fee2-0e27-4b24-9bb2-360e2d851e1c admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:36:13.961990 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1330/2660] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:36:13 2022] PUT /identity/v3/projects/f69d85fa8dc545efb55a0b2949e9f06c/users/ffe9a464991e4965a0638d0c23b17ae7/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:36:13.968046 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ce06d-49aa-4da1-9df8-96e90bd650d9 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:36:13.968334 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ce06d-49aa-4da1-9df8-96e90bd650d9 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:36:13.968606 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ce06d-49aa-4da1-9df8-96e90bd650d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:36:13.987002 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-5d4ce06d-49aa-4da1-9df8-96e90bd650d9 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:36:13.988204 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-5d4ce06d-49aa-4da1-9df8-96e90bd650d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:36:14.045861 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1331/2661] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:36:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:36:14.055428 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac2fee2-0e27-4b24-9bb2-360e2d851e1c admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:36:14.057489 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-3ac2fee2-0e27-4b24-9bb2-360e2d851e1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:36:14.099645 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:36:14.116087 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac2fee2-0e27-4b24-9bb2-360e2d851e1c admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:36:14.119673 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf64e4-462b-4e58-afd6-275aa43e2fd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:36:14.120258 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf64e4-462b-4e58-afd6-275aa43e2fd1 service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:36:14.120532 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf64e4-462b-4e58-afd6-275aa43e2fd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:36:14.120797 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf64e4-462b-4e58-afd6-275aa43e2fd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:36:14.130004 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bf64e4-462b-4e58-afd6-275aa43e2fd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:36:14.131462 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bf64e4-462b-4e58-afd6-275aa43e2fd1 service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:36:14.140710 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1331/2662] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:36:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:36:16.767670 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ce06d-49aa-4da1-9df8-96e90bd650d9 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:36:16.769980 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-5d4ce06d-49aa-4da1-9df8-96e90bd650d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:36:16.786173 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ce06d-49aa-4da1-9df8-96e90bd650d9 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:36:16.790350 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8987a-b9c8-468a-9c94-20dc70002aff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:36:16.791198 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8987a-b9c8-468a-9c94-20dc70002aff service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:36:16.791721 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8987a-b9c8-468a-9c94-20dc70002aff service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:36:16.792184 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8987a-b9c8-468a-9c94-20dc70002aff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:36:16.794777 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-92c8987a-b9c8-468a-9c94-20dc70002aff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:36:16.892320 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c8987a-b9c8-468a-9c94-20dc70002aff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:36:16.906446 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c8987a-b9c8-468a-9c94-20dc70002aff service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:36:16.928664 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1332/2663] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:36:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:36:16.996166 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf64e4-462b-4e58-afd6-275aa43e2fd1 service neutron] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:36:16.998932 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-63bf64e4-462b-4e58-afd6-275aa43e2fd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:36:17.102961 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf64e4-462b-4e58-afd6-275aa43e2fd1 service neutron] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:36:17.108447 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2777dc-a61a-45e3-a4b0-c81b9246615a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e82b01dbd6e84a9d82baa2c28d77fc28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:36:17.109745 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2777dc-a61a-45e3-a4b0-c81b9246615a service glance] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:36:17.109745 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2777dc-a61a-45e3-a4b0-c81b9246615a service glance] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:36:17.110081 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2777dc-a61a-45e3-a4b0-c81b9246615a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:36:17.112053 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-6d2777dc-a61a-45e3-a4b0-c81b9246615a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:36:17.135925 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2777dc-a61a-45e3-a4b0-c81b9246615a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:36:17.145645 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2777dc-a61a-45e3-a4b0-c81b9246615a service glance] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:36:17.161435 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1332/2664] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:36:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:36:18.404284 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcae751-8be0-4d8e-92a0-c2c2f46f591c None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:36:18.405309 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcae751-8be0-4d8e-92a0-c2c2f46f591c None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:36:18.405949 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcae751-8be0-4d8e-92a0-c2c2f46f591c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:36:18.451557 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-7dcae751-8be0-4d8e-92a0-c2c2f46f591c None None] MFA Rules not processed for user `ffe9a464991e4965a0638d0c23b17ae7`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:36:18.452723 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-7dcae751-8be0-4d8e-92a0-c2c2f46f591c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:36:18.532741 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1333/2665] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:36:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:36:18.539139 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2777dc-a61a-45e3-a4b0-c81b9246615a service glance] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:36:18.555267 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2777dc-a61a-45e3-a4b0-c81b9246615a service glance] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:36:18.559933 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4852368e-dacb-4c68-af47-17652c7fe176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:36:18.560545 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852368e-dacb-4c68-af47-17652c7fe176 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:36:18.560820 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852368e-dacb-4c68-af47-17652c7fe176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:36:18.561231 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852368e-dacb-4c68-af47-17652c7fe176 admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:36:18.561872 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4852368e-dacb-4c68-af47-17652c7fe176 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:36:18.563835 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4852368e-dacb-4c68-af47-17652c7fe176 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:36:18.594480 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1333/2666] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:36:18 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:36:18.598541 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcae751-8be0-4d8e-92a0-c2c2f46f591c None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:36:18.611175 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcae751-8be0-4d8e-92a0-c2c2f46f591c None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:36:18.617242 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:36:18.679843 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8830ccf0-db6e-4d4f-9be9-b8435e9c3077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:36:18.680383 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8830ccf0-db6e-4d4f-9be9-b8435e9c3077 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:36:18.680653 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8830ccf0-db6e-4d4f-9be9-b8435e9c3077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:36:18.680918 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8830ccf0-db6e-4d4f-9be9-b8435e9c3077 admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:36:18.681564 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8830ccf0-db6e-4d4f-9be9-b8435e9c3077 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:36:18.683516 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8830ccf0-db6e-4d4f-9be9-b8435e9c3077 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:36:18.710867 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1334/2667] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:36:18 2022] POST /identity/v3/users => generated 295 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:36:18.712134 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4852368e-dacb-4c68-af47-17652c7fe176 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:36:18.726049 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4852368e-dacb-4c68-af47-17652c7fe176 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:36:18.731629 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f81643-4536-4e85-97fa-502546c57d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:36:18.732247 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f81643-4536-4e85-97fa-502546c57d0e admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:36:18.732522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f81643-4536-4e85-97fa-502546c57d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:36:18.733040 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f81643-4536-4e85-97fa-502546c57d0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:36:18.733444 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f81643-4536-4e85-97fa-502546c57d0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:36:18.735087 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f81643-4536-4e85-97fa-502546c57d0e admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:36:18.747036 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1334/2668] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 14:36:18.751471 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8830ccf0-db6e-4d4f-9be9-b8435e9c3077 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:36:18.762644 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8830ccf0-db6e-4d4f-9be9-b8435e9c3077 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:36:18.768449 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d959f81-5957-405e-af39-7205e529f559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:36:18.769127 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d959f81-5957-405e-af39-7205e529f559 admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:36:18.769504 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d959f81-5957-405e-af39-7205e529f559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:36:18.769789 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d959f81-5957-405e-af39-7205e529f559 admin admin] PATH_INFO: `/v3/projects/4039b94848584573b523d0c3d844c47d/users/07def5ab50ed44e1abf49130288c222d/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:36:18.789302 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d959f81-5957-405e-af39-7205e529f559 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4039b94848584573b523d0c3d844c47d, user_id=07def5ab50ed44e1abf49130288c222d, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:36:18.792280 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d959f81-5957-405e-af39-7205e529f559 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:36:18.822727 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1335/2669] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:36:18 2022] PUT /identity/v3/projects/4039b94848584573b523d0c3d844c47d/users/07def5ab50ed44e1abf49130288c222d/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:36:18.827199 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f81643-4536-4e85-97fa-502546c57d0e admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:36:18.841187 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f81643-4536-4e85-97fa-502546c57d0e admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:36:18.903448 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1a7d4-2f90-448b-8716-668d5326e138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:36:18.904076 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1a7d4-2f90-448b-8716-668d5326e138 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:36:18.904365 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1a7d4-2f90-448b-8716-668d5326e138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:36:18.904638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1a7d4-2f90-448b-8716-668d5326e138 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:36:18.905280 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef1a7d4-2f90-448b-8716-668d5326e138 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:36:18.907095 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef1a7d4-2f90-448b-8716-668d5326e138 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:36:18.920293 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1335/2670] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:36:18.924743 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d959f81-5957-405e-af39-7205e529f559 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:36:18.938182 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d959f81-5957-405e-af39-7205e529f559 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:36:18.952039 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549d563-9ca3-4f8e-94e2-cea45d7ac46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:36:18.952724 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549d563-9ca3-4f8e-94e2-cea45d7ac46e admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:36:18.953003 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549d563-9ca3-4f8e-94e2-cea45d7ac46e admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:36:18.953279 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549d563-9ca3-4f8e-94e2-cea45d7ac46e admin admin] PATH_INFO: `/v3/projects/4039b94848584573b523d0c3d844c47d/users/07def5ab50ed44e1abf49130288c222d/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:36:18.955256 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549d563-9ca3-4f8e-94e2-cea45d7ac46e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4039b94848584573b523d0c3d844c47d, user_id=07def5ab50ed44e1abf49130288c222d, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:36:18.957945 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549d563-9ca3-4f8e-94e2-cea45d7ac46e admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:36:18.969247 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1336/2671] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:36:18 2022] PUT /identity/v3/projects/4039b94848584573b523d0c3d844c47d/users/07def5ab50ed44e1abf49130288c222d/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:36:22.458358 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1a7d4-2f90-448b-8716-668d5326e138 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:36:22.460260 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-bef1a7d4-2f90-448b-8716-668d5326e138 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:36:22.556461 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:36:22.598909 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1a7d4-2f90-448b-8716-668d5326e138 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:36:22.743560 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad70f6-cf40-412b-97d3-59d3745e27b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:36:22.744338 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad70f6-cf40-412b-97d3-59d3745e27b1 service nova] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:36:22.744838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad70f6-cf40-412b-97d3-59d3745e27b1 service nova] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:36:22.745165 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad70f6-cf40-412b-97d3-59d3745e27b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:36:22.748675 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-adad70f6-cf40-412b-97d3-59d3745e27b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:36:22.872325 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adad70f6-cf40-412b-97d3-59d3745e27b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:36:22.873733 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adad70f6-cf40-412b-97d3-59d3745e27b1 service nova] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:36:22.906525 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1336/2672] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:36:22 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:36:23.834290 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cacd7f-15cd-4280-8597-e5b21f2b27a8 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:36:23.834828 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cacd7f-15cd-4280-8597-e5b21f2b27a8 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:36:23.834953 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cacd7f-15cd-4280-8597-e5b21f2b27a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:36:23.863696 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-15cacd7f-15cd-4280-8597-e5b21f2b27a8 None None] MFA Rules not processed for user `07def5ab50ed44e1abf49130288c222d`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:36:23.864825 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-15cacd7f-15cd-4280-8597-e5b21f2b27a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:36:23.934632 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1337/2673] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:36:23 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:36:23.948258 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad70f6-cf40-412b-97d3-59d3745e27b1 service nova] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:36:23.960838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad70f6-cf40-412b-97d3-59d3745e27b1 service nova] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:36:23.968207 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6592fd65-c81f-4854-ae3f-97f0951c7c28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:36:23.968920 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592fd65-c81f-4854-ae3f-97f0951c7c28 service nova] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:36:23.969267 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592fd65-c81f-4854-ae3f-97f0951c7c28 service nova] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:36:23.969605 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592fd65-c81f-4854-ae3f-97f0951c7c28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:36:24.017602 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:36:24.026390 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6592fd65-c81f-4854-ae3f-97f0951c7c28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:36:24.029081 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6592fd65-c81f-4854-ae3f-97f0951c7c28 service nova] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:36:24.038278 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1337/2674] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:36:23 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:36:33.931753 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=74,keystone:INSERT=5 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:36:34.034156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=4 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:36:38.048937 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89173bac-830f-428c-8a04-54fdf950e900 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:36:38.049412 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89173bac-830f-428c-8a04-54fdf950e900 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:36:38.049524 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89173bac-830f-428c-8a04-54fdf950e900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:36:38.099862 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-89173bac-830f-428c-8a04-54fdf950e900 None None] MFA Rules not processed for user `82328b7fd50649a485c6eaf27c5597c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:36:38.101054 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-89173bac-830f-428c-8a04-54fdf950e900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:36:38.166802 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1338/2675] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 14:36:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:36:38.179618 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6592fd65-c81f-4854-ae3f-97f0951c7c28 service nova] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:36:38.181179 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-6592fd65-c81f-4854-ae3f-97f0951c7c28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:36:38.271984 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6592fd65-c81f-4854-ae3f-97f0951c7c28 service nova] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:36:38.276939 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5526c432-3b89-4419-83b1-8a04b463f22d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '879f296733114f6f844a98aaefe35e92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:36:38.279644 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526c432-3b89-4419-83b1-8a04b463f22d service swift] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:36:38.279922 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526c432-3b89-4419-83b1-8a04b463f22d service swift] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:36:38.280196 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526c432-3b89-4419-83b1-8a04b463f22d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:36:38.288317 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5526c432-3b89-4419-83b1-8a04b463f22d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:36:38.289850 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5526c432-3b89-4419-83b1-8a04b463f22d service swift] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:36:38.299024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1338/2676] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 14:36:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 14:36:48.163684 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=40 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:36:48.295743 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=41 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:36:50.318935 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be4d7b-24d3-4167-be2c-4ff9aa7f726f None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:36:50.319443 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be4d7b-24d3-4167-be2c-4ff9aa7f726f None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:36:50.319537 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be4d7b-24d3-4167-be2c-4ff9aa7f726f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:36:50.363930 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-03be4d7b-24d3-4167-be2c-4ff9aa7f726f None None] MFA Rules not processed for user `ffe9a464991e4965a0638d0c23b17ae7`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:36:50.366668 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-03be4d7b-24d3-4167-be2c-4ff9aa7f726f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:36:50.379823 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1339/2677] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:36:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:36:51.274053 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5526c432-3b89-4419-83b1-8a04b463f22d service swift] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:36:51.276097 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-5526c432-3b89-4419-83b1-8a04b463f22d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:36:51.383725 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5526c432-3b89-4419-83b1-8a04b463f22d service swift] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:36:51.389239 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b37dde1-3d67-4595-b7e8-d68411e501ca service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18fe384ecd454fb298dcc788f41dd44b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:36:51.389870 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b37dde1-3d67-4595-b7e8-d68411e501ca service placement] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:36:51.390147 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b37dde1-3d67-4595-b7e8-d68411e501ca service placement] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:36:51.390418 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b37dde1-3d67-4595-b7e8-d68411e501ca service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:36:51.392256 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-7b37dde1-3d67-4595-b7e8-d68411e501ca service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:36:51.405809 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b37dde1-3d67-4595-b7e8-d68411e501ca service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:36:51.407566 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b37dde1-3d67-4595-b7e8-d68411e501ca service placement] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:36:51.419385 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1339/2678] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:36:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:36:53.726547 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be4d7b-24d3-4167-be2c-4ff9aa7f726f None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:36:53.784544 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be4d7b-24d3-4167-be2c-4ff9aa7f726f None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:36:53.788160 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1631b077-bc39-4920-9754-cf252e91819d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:36:53.788760 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631b077-bc39-4920-9754-cf252e91819d service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:36:53.789035 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631b077-bc39-4920-9754-cf252e91819d service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:36:53.789303 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631b077-bc39-4920-9754-cf252e91819d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:36:53.791062 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-1631b077-bc39-4920-9754-cf252e91819d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:36:53.801362 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1631b077-bc39-4920-9754-cf252e91819d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:36:53.802772 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1631b077-bc39-4920-9754-cf252e91819d service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:36:53.811568 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1340/2679] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:36:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 14:36:56.656944 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b37dde1-3d67-4595-b7e8-d68411e501ca service placement] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:36:56.673891 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b37dde1-3d67-4595-b7e8-d68411e501ca service placement] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:36:56.679475 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291faa7f-fe03-4382-8f48-9d8beb102757 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:36:56.680082 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291faa7f-fe03-4382-8f48-9d8beb102757 service nova] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:36:56.680363 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291faa7f-fe03-4382-8f48-9d8beb102757 service nova] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:36:56.681370 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291faa7f-fe03-4382-8f48-9d8beb102757 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:36:56.683503 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-291faa7f-fe03-4382-8f48-9d8beb102757 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:36:56.696192 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291faa7f-fe03-4382-8f48-9d8beb102757 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:36:56.697770 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291faa7f-fe03-4382-8f48-9d8beb102757 service nova] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:36:56.708970 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1340/2680] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:36:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:37:00.274006 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2b3c9-147b-4977-b2c7-b6925066a10e None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:37:00.274537 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2b3c9-147b-4977-b2c7-b6925066a10e None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:37:00.274635 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2b3c9-147b-4977-b2c7-b6925066a10e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:37:00.295788 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-b3c2b3c9-147b-4977-b2c7-b6925066a10e None None] MFA Rules not processed for user `82328b7fd50649a485c6eaf27c5597c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:37:00.297000 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-b3c2b3c9-147b-4977-b2c7-b6925066a10e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:37:00.304521 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1341/2681] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 14:37:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:37:00.316910 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291faa7f-fe03-4382-8f48-9d8beb102757 service nova] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:37:00.329028 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291faa7f-fe03-4382-8f48-9d8beb102757 service nova] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:37:00.334080 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2710b19-f87d-48ed-a33a-83965ff49b69 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '879f296733114f6f844a98aaefe35e92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:37:00.334688 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2710b19-f87d-48ed-a33a-83965ff49b69 service swift] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:37:00.335376 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2710b19-f87d-48ed-a33a-83965ff49b69 service swift] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:37:00.335376 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2710b19-f87d-48ed-a33a-83965ff49b69 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:37:00.344655 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2710b19-f87d-48ed-a33a-83965ff49b69 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:37:00.346237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2710b19-f87d-48ed-a33a-83965ff49b69 service swift] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:37:00.354464 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1341/2682] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 14:37:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 14:37:10.290578 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=32 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:37:10.351549 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=44 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:37:18.548770 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c2b3c9-147b-4977-b2c7-b6925066a10e None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:37:18.565567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c2b3c9-147b-4977-b2c7-b6925066a10e None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:37:18.571439 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7496f6e7-6b53-434f-bd63-e57a69d37364 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:37:18.572049 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496f6e7-6b53-434f-bd63-e57a69d37364 service nova] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:37:18.572879 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496f6e7-6b53-434f-bd63-e57a69d37364 service nova] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:37:18.572879 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496f6e7-6b53-434f-bd63-e57a69d37364 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:37:18.576650 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-7496f6e7-6b53-434f-bd63-e57a69d37364 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:37:18.589694 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7496f6e7-6b53-434f-bd63-e57a69d37364 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:37:18.591895 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7496f6e7-6b53-434f-bd63-e57a69d37364 service nova] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:37:18.603188 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1342/2683] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:37:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:37:20.898730 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa519d-623a-43dd-b17a-49c0af2ae0c1 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:37:20.899284 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa519d-623a-43dd-b17a-49c0af2ae0c1 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:37:20.899284 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa519d-623a-43dd-b17a-49c0af2ae0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:37:20.922128 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-bafa519d-623a-43dd-b17a-49c0af2ae0c1 None None] MFA Rules not processed for user `82328b7fd50649a485c6eaf27c5597c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:37:20.924114 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-bafa519d-623a-43dd-b17a-49c0af2ae0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:37:20.932572 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1342/2684] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 14:37:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:37:20.972228 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7496f6e7-6b53-434f-bd63-e57a69d37364 service nova] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:37:20.984950 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7496f6e7-6b53-434f-bd63-e57a69d37364 service nova] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:37:20.991216 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa83fb29-f7ed-4d72-9b12-b65938bc671b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '879f296733114f6f844a98aaefe35e92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:37:20.992057 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa83fb29-f7ed-4d72-9b12-b65938bc671b service swift] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:37:20.992331 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa83fb29-f7ed-4d72-9b12-b65938bc671b service swift] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:37:20.992594 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa83fb29-f7ed-4d72-9b12-b65938bc671b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:37:21.001203 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa83fb29-f7ed-4d72-9b12-b65938bc671b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:37:21.002769 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa83fb29-f7ed-4d72-9b12-b65938bc671b service swift] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:37:21.011167 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1343/2685] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 14:37:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 14:37:23.591340 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa519d-623a-43dd-b17a-49c0af2ae0c1 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:37:23.601248 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa519d-623a-43dd-b17a-49c0af2ae0c1 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:37:23.604741 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e454ebe-0193-4a0e-ace4-34f55650e4a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:37:23.605354 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e454ebe-0193-4a0e-ace4-34f55650e4a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:37:23.605628 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e454ebe-0193-4a0e-ace4-34f55650e4a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:37:23.605894 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e454ebe-0193-4a0e-ace4-34f55650e4a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:37:23.614665 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e454ebe-0193-4a0e-ace4-34f55650e4a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:37:23.616111 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e454ebe-0193-4a0e-ace4-34f55650e4a0 service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:37:23.626154 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1343/2686] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:37:23 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:37:26.603898 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e601a-a151-4c0b-a624-194837ab2c38 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:37:26.604555 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e601a-a151-4c0b-a624-194837ab2c38 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:37:26.604692 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e601a-a151-4c0b-a624-194837ab2c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:37:26.624882 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-2d1e601a-a151-4c0b-a624-194837ab2c38 None None] MFA Rules not processed for user `abb529f39710407abf9d6e1cedd9a603`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:37:26.626065 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-2d1e601a-a151-4c0b-a624-194837ab2c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:37:26.679313 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1344/2687] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:37:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:37:33.621281 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=9 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:37:36.672469 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=37 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:38:26.276578 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e454ebe-0193-4a0e-ace4-34f55650e4a0 service neutron] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:38:26.286148 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e454ebe-0193-4a0e-ace4-34f55650e4a0 service neutron] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:38:26.289809 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4794ab-a59b-4d30-b49f-285ecc429645 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:38:26.290405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4794ab-a59b-4d30-b49f-285ecc429645 service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:38:26.290681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4794ab-a59b-4d30-b49f-285ecc429645 service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:38:26.290992 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4794ab-a59b-4d30-b49f-285ecc429645 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:38:26.292708 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-9f4794ab-a59b-4d30-b49f-285ecc429645 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:38:26.349727 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4794ab-a59b-4d30-b49f-285ecc429645 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:38:26.351194 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4794ab-a59b-4d30-b49f-285ecc429645 service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:38:26.359794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1344/2688] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:38:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:38:33.694732 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e601a-a151-4c0b-a624-194837ab2c38 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:38:33.704769 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e601a-a151-4c0b-a624-194837ab2c38 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:38:33.709107 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e39d8f-369e-402d-908c-bc224f610c0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:38:33.710097 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e39d8f-369e-402d-908c-bc224f610c0f service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:38:33.710438 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e39d8f-369e-402d-908c-bc224f610c0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:38:33.710832 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e39d8f-369e-402d-908c-bc224f610c0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:38:33.713690 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-96e39d8f-369e-402d-908c-bc224f610c0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:38:33.794629 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e39d8f-369e-402d-908c-bc224f610c0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:38:33.796794 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e39d8f-369e-402d-908c-bc224f610c0f service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:38:33.806907 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1345/2689] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:38:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:38:36.355878 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=28 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:38:36.739102 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4794ab-a59b-4d30-b49f-285ecc429645 service neutron] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:38:36.740851 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-9f4794ab-a59b-4d30-b49f-285ecc429645 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:38:36.757231 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4794ab-a59b-4d30-b49f-285ecc429645 service neutron] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:38:36.762223 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a5f17-cce3-489d-9204-65f69cf69e02 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18fe384ecd454fb298dcc788f41dd44b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:38:36.762953 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a5f17-cce3-489d-9204-65f69cf69e02 service placement] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:38:36.763226 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a5f17-cce3-489d-9204-65f69cf69e02 service placement] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:38:36.763496 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a5f17-cce3-489d-9204-65f69cf69e02 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:38:36.765758 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-818a5f17-cce3-489d-9204-65f69cf69e02 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:38:36.781917 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818a5f17-cce3-489d-9204-65f69cf69e02 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:38:36.784022 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818a5f17-cce3-489d-9204-65f69cf69e02 service placement] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:38:36.811133 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1345/2690] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:38:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:38:38.293360 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e39d8f-369e-402d-908c-bc224f610c0f service neutron] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:38:38.302337 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e39d8f-369e-402d-908c-bc224f610c0f service neutron] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:38:38.306142 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2b8bc-0d30-458f-aa84-b4d8dafd6da3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:38:38.306837 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2b8bc-0d30-458f-aa84-b4d8dafd6da3 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:38:38.307138 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2b8bc-0d30-458f-aa84-b4d8dafd6da3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:38:38.307413 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2b8bc-0d30-458f-aa84-b4d8dafd6da3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:38:38.309185 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-3aa2b8bc-0d30-458f-aa84-b4d8dafd6da3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:38:38.323146 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa2b8bc-0d30-458f-aa84-b4d8dafd6da3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:38:38.324651 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa2b8bc-0d30-458f-aa84-b4d8dafd6da3 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:38:38.338586 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1346/2691] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:38:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:38:40.563052 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a5f17-cce3-489d-9204-65f69cf69e02 service placement] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:38:40.565795 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-818a5f17-cce3-489d-9204-65f69cf69e02 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:38:40.580707 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a5f17-cce3-489d-9204-65f69cf69e02 service placement] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:38:40.589599 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a1abd-d1f7-454d-99c2-625012db7297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:38:40.590444 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a1abd-d1f7-454d-99c2-625012db7297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:38:40.590792 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a1abd-d1f7-454d-99c2-625012db7297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:38:40.591160 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a1abd-d1f7-454d-99c2-625012db7297 admin admin] PATH_INFO: `/v3/users/e4c9bfcd6d1a401194a7176f8aa2e6ac` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:38:40.592373 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58a1abd-d1f7-454d-99c2-625012db7297 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4c9bfcd6d1a401194a7176f8aa2e6ac)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:38:40.594641 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58a1abd-d1f7-454d-99c2-625012db7297 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:38:40.682053 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-d58a1abd-d1f7-454d-99c2-625012db7297 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4c9bfcd6d1a401194a7176f8aa2e6ac'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:38:40.683667 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-d58a1abd-d1f7-454d-99c2-625012db7297 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4c9bfcd6d1a401194a7176f8aa2e6ac'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:38:40.691908 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-d58a1abd-d1f7-454d-99c2-625012db7297 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4c9bfcd6d1a401194a7176f8aa2e6ac'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:38:40.692598 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-d58a1abd-d1f7-454d-99c2-625012db7297 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4c9bfcd6d1a401194a7176f8aa2e6ac'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:38:40.707431 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-d58a1abd-d1f7-454d-99c2-625012db7297 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4c9bfcd6d1a401194a7176f8aa2e6ac'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:38:40.722460 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1346/2692] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:38:40 2022] DELETE /identity/v3/users/e4c9bfcd6d1a401194a7176f8aa2e6ac => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:38:40.914531 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2b8bc-0d30-458f-aa84-b4d8dafd6da3 service neutron] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:38:40.933837 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2b8bc-0d30-458f-aa84-b4d8dafd6da3 service neutron] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:38:40.996160 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877ec34-c6fe-4c10-9f02-4de8273ad6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:38:40.996792 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877ec34-c6fe-4c10-9f02-4de8273ad6e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:38:40.997325 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877ec34-c6fe-4c10-9f02-4de8273ad6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:38:40.997601 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877ec34-c6fe-4c10-9f02-4de8273ad6e0 admin admin] PATH_INFO: `/v3/projects/71699c60daeb48aaaf5b0611c8f68bc4` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:38:40.998794 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3877ec34-c6fe-4c10-9f02-4de8273ad6e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71699c60daeb48aaaf5b0611c8f68bc4)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:38:41.000739 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3877ec34-c6fe-4c10-9f02-4de8273ad6e0 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:38:41.049114 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-3877ec34-c6fe-4c10-9f02-4de8273ad6e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71699c60daeb48aaaf5b0611c8f68bc4', 'request_id': 'req-3877ec34-c6fe-4c10-9f02-4de8273ad6e0'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:38:41.056408 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-3877ec34-c6fe-4c10-9f02-4de8273ad6e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71699c60daeb48aaaf5b0611c8f68bc4', 'request_id': 'req-3877ec34-c6fe-4c10-9f02-4de8273ad6e0'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:38:41.067584 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-3877ec34-c6fe-4c10-9f02-4de8273ad6e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71699c60daeb48aaaf5b0611c8f68bc4', 'request_id': 'req-3877ec34-c6fe-4c10-9f02-4de8273ad6e0'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:38:41.084248 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-3877ec34-c6fe-4c10-9f02-4de8273ad6e0 admin admin] The token cache is being invalidate because project 71699c60daeb48aaaf5b0611c8f68bc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:38:41.084620 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-3877ec34-c6fe-4c10-9f02-4de8273ad6e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:38:41.085225 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-3877ec34-c6fe-4c10-9f02-4de8273ad6e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:38:41.093563 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1347/2693] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:38:40 2022] DELETE /identity/v3/projects/71699c60daeb48aaaf5b0611c8f68bc4 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:38:41.096751 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a1abd-d1f7-454d-99c2-625012db7297 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:38:41.096751 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-d58a1abd-d1f7-454d-99c2-625012db7297 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:38:41.111966 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a1abd-d1f7-454d-99c2-625012db7297 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:38:41.171072 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a129d8e0-3496-4cbf-b3fb-3183fac0a172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:38:41.171996 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a129d8e0-3496-4cbf-b3fb-3183fac0a172 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:38:41.172276 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a129d8e0-3496-4cbf-b3fb-3183fac0a172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:38:41.172592 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a129d8e0-3496-4cbf-b3fb-3183fac0a172 admin admin] PATH_INFO: `/v3/users/b8bbed6164f24411aaa0054004c9d3ad` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:38:41.174465 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a129d8e0-3496-4cbf-b3fb-3183fac0a172 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8bbed6164f24411aaa0054004c9d3ad)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:38:41.176497 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a129d8e0-3496-4cbf-b3fb-3183fac0a172 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:38:41.226093 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-a129d8e0-3496-4cbf-b3fb-3183fac0a172 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8bbed6164f24411aaa0054004c9d3ad'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:38:41.226737 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-a129d8e0-3496-4cbf-b3fb-3183fac0a172 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8bbed6164f24411aaa0054004c9d3ad'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:38:41.233868 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-a129d8e0-3496-4cbf-b3fb-3183fac0a172 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8bbed6164f24411aaa0054004c9d3ad'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:38:41.234492 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-a129d8e0-3496-4cbf-b3fb-3183fac0a172 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8bbed6164f24411aaa0054004c9d3ad'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:38:41.238918 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:38:41.248640 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-a129d8e0-3496-4cbf-b3fb-3183fac0a172 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8bbed6164f24411aaa0054004c9d3ad'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:38:41.263449 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1347/2694] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:38:41 2022] DELETE /identity/v3/users/b8bbed6164f24411aaa0054004c9d3ad => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:38:41.453616 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877ec34-c6fe-4c10-9f02-4de8273ad6e0 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:38:41.455494 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-3877ec34-c6fe-4c10-9f02-4de8273ad6e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:38:41.487750 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877ec34-c6fe-4c10-9f02-4de8273ad6e0 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:38:41.499404 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:38:41.563280 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8118090c-a3fb-4823-a60d-fef2436f3a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:38:41.563914 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8118090c-a3fb-4823-a60d-fef2436f3a6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:38:41.564192 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8118090c-a3fb-4823-a60d-fef2436f3a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:38:41.564471 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8118090c-a3fb-4823-a60d-fef2436f3a6c admin admin] PATH_INFO: `/v3/projects/88e9d7caf24e4ed288bf8b152538daeb` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:38:41.565713 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8118090c-a3fb-4823-a60d-fef2436f3a6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=88e9d7caf24e4ed288bf8b152538daeb)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:38:41.567646 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8118090c-a3fb-4823-a60d-fef2436f3a6c admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:38:41.627398 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-8118090c-a3fb-4823-a60d-fef2436f3a6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88e9d7caf24e4ed288bf8b152538daeb', 'request_id': 'req-8118090c-a3fb-4823-a60d-fef2436f3a6c'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:38:41.634122 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-8118090c-a3fb-4823-a60d-fef2436f3a6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88e9d7caf24e4ed288bf8b152538daeb', 'request_id': 'req-8118090c-a3fb-4823-a60d-fef2436f3a6c'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:38:41.642654 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-8118090c-a3fb-4823-a60d-fef2436f3a6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88e9d7caf24e4ed288bf8b152538daeb', 'request_id': 'req-8118090c-a3fb-4823-a60d-fef2436f3a6c'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:38:41.656687 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-8118090c-a3fb-4823-a60d-fef2436f3a6c admin admin] The token cache is being invalidate because project 88e9d7caf24e4ed288bf8b152538daeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:38:41.657057 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-8118090c-a3fb-4823-a60d-fef2436f3a6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:38:41.657641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-8118090c-a3fb-4823-a60d-fef2436f3a6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:38:41.659445 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1348/2695] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:38:41 2022] DELETE /identity/v3/projects/88e9d7caf24e4ed288bf8b152538daeb => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:38:42.180517 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a129d8e0-3496-4cbf-b3fb-3183fac0a172 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:38:42.182105 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-a129d8e0-3496-4cbf-b3fb-3183fac0a172 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:38:42.199097 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a129d8e0-3496-4cbf-b3fb-3183fac0a172 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:38:42.255422 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4171b2-2c15-4cea-9b54-3baf5face689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:38:42.256054 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4171b2-2c15-4cea-9b54-3baf5face689 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:38:42.256336 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4171b2-2c15-4cea-9b54-3baf5face689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:38:42.256609 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4171b2-2c15-4cea-9b54-3baf5face689 admin admin] PATH_INFO: `/v3/users/883e6570afb84a099e46cdace5a88b2e` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:38:42.257715 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4171b2-2c15-4cea-9b54-3baf5face689 admin admin] RBAC: Authorizing `identity:delete_user(user_id=883e6570afb84a099e46cdace5a88b2e)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:38:42.259674 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4171b2-2c15-4cea-9b54-3baf5face689 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:38:42.304786 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-fa4171b2-2c15-4cea-9b54-3baf5face689 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '883e6570afb84a099e46cdace5a88b2e'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:38:42.305459 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-fa4171b2-2c15-4cea-9b54-3baf5face689 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '883e6570afb84a099e46cdace5a88b2e'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:38:42.313177 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-fa4171b2-2c15-4cea-9b54-3baf5face689 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '883e6570afb84a099e46cdace5a88b2e'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:38:42.313805 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-fa4171b2-2c15-4cea-9b54-3baf5face689 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '883e6570afb84a099e46cdace5a88b2e'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:38:42.325874 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-fa4171b2-2c15-4cea-9b54-3baf5face689 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '883e6570afb84a099e46cdace5a88b2e'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:38:42.338375 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1348/2696] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:38:42 2022] DELETE /identity/v3/users/883e6570afb84a099e46cdace5a88b2e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:38:42.507287 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8118090c-a3fb-4823-a60d-fef2436f3a6c admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:38:42.509547 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-8118090c-a3fb-4823-a60d-fef2436f3a6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:38:42.528236 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8118090c-a3fb-4823-a60d-fef2436f3a6c admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:38:42.595597 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd59e55a-5f66-42bc-97ff-09cd444f799a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:38:42.596229 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd59e55a-5f66-42bc-97ff-09cd444f799a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:38:42.596509 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd59e55a-5f66-42bc-97ff-09cd444f799a admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:38:42.596786 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd59e55a-5f66-42bc-97ff-09cd444f799a admin admin] PATH_INFO: `/v3/projects/1f7134ad65014cb984417c99b3d76eff` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:38:42.597897 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd59e55a-5f66-42bc-97ff-09cd444f799a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f7134ad65014cb984417c99b3d76eff)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:38:42.599891 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd59e55a-5f66-42bc-97ff-09cd444f799a admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:38:42.642280 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:38:42.646579 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-dd59e55a-5f66-42bc-97ff-09cd444f799a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f7134ad65014cb984417c99b3d76eff', 'request_id': 'req-dd59e55a-5f66-42bc-97ff-09cd444f799a'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:38:42.654276 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-dd59e55a-5f66-42bc-97ff-09cd444f799a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f7134ad65014cb984417c99b3d76eff', 'request_id': 'req-dd59e55a-5f66-42bc-97ff-09cd444f799a'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:38:42.660638 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-dd59e55a-5f66-42bc-97ff-09cd444f799a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f7134ad65014cb984417c99b3d76eff', 'request_id': 'req-dd59e55a-5f66-42bc-97ff-09cd444f799a'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:38:42.672117 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-dd59e55a-5f66-42bc-97ff-09cd444f799a admin admin] The token cache is being invalidate because project 1f7134ad65014cb984417c99b3d76eff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:38:42.672399 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-dd59e55a-5f66-42bc-97ff-09cd444f799a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:38:42.673011 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-dd59e55a-5f66-42bc-97ff-09cd444f799a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:38:42.674448 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1349/2697] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:38:42 2022] DELETE /identity/v3/projects/1f7134ad65014cb984417c99b3d76eff => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:38:42.679489 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97151b-b04f-4481-8a9c-66c77006afe2 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:38:42.679793 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97151b-b04f-4481-8a9c-66c77006afe2 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:38:42.680063 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97151b-b04f-4481-8a9c-66c77006afe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:38:42.700139 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-bf97151b-b04f-4481-8a9c-66c77006afe2 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:38:42.701827 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-bf97151b-b04f-4481-8a9c-66c77006afe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:38:42.751170 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:38:42.761794 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1349/2698] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:38:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:38:42.770830 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104a2d0-be4a-4c65-b06b-8d8c52410d33 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:38:42.771141 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104a2d0-be4a-4c65-b06b-8d8c52410d33 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:38:42.771435 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104a2d0-be4a-4c65-b06b-8d8c52410d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:38:42.791493 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-8104a2d0-be4a-4c65-b06b-8d8c52410d33 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:38:42.792712 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-8104a2d0-be4a-4c65-b06b-8d8c52410d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:38:42.800617 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1350/2699] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:38:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:38:42.804716 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf97151b-b04f-4481-8a9c-66c77006afe2 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:38:42.814803 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf97151b-b04f-4481-8a9c-66c77006afe2 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:38:42.819733 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29dfc81-1ff1-4496-8c41-a8658135f528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:38:42.820342 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29dfc81-1ff1-4496-8c41-a8658135f528 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:38:42.820727 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29dfc81-1ff1-4496-8c41-a8658135f528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:38:42.821031 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29dfc81-1ff1-4496-8c41-a8658135f528 admin admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:38:42.821739 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29dfc81-1ff1-4496-8c41-a8658135f528 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:38:42.822043 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29dfc81-1ff1-4496-8c41-a8658135f528 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:38:42.823895 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29dfc81-1ff1-4496-8c41-a8658135f528 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:38:42.836356 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1350/2700] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:38:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:38:42.840634 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8104a2d0-be4a-4c65-b06b-8d8c52410d33 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:38:42.850374 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8104a2d0-be4a-4c65-b06b-8d8c52410d33 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:38:42.855065 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e206444-167d-4f32-8ece-0b13f8708890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:38:42.855660 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e206444-167d-4f32-8ece-0b13f8708890 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:38:42.855941 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e206444-167d-4f32-8ece-0b13f8708890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:38:42.856215 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e206444-167d-4f32-8ece-0b13f8708890 admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:38:42.856853 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e206444-167d-4f32-8ece-0b13f8708890 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:38:42.858769 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e206444-167d-4f32-8ece-0b13f8708890 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:38:42.884814 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1351/2701] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:38:42 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:38:42.889321 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29dfc81-1ff1-4496-8c41-a8658135f528 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:38:42.899338 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29dfc81-1ff1-4496-8c41-a8658135f528 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:38:42.954953 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763ef198-f64a-4797-b96c-43c4722c9c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:38:42.955469 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ef198-f64a-4797-b96c-43c4722c9c5a admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:38:42.955746 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ef198-f64a-4797-b96c-43c4722c9c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:38:42.956014 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ef198-f64a-4797-b96c-43c4722c9c5a admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:38:42.956639 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763ef198-f64a-4797-b96c-43c4722c9c5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:38:42.958540 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763ef198-f64a-4797-b96c-43c4722c9c5a admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:38:42.979178 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1351/2702] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:38:42 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:38:42.983303 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e206444-167d-4f32-8ece-0b13f8708890 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:38:42.993271 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e206444-167d-4f32-8ece-0b13f8708890 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:38:42.997786 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bec2ed-447c-4dfd-8281-33cfb8e991cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:38:42.998888 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bec2ed-447c-4dfd-8281-33cfb8e991cd admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:38:42.999164 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bec2ed-447c-4dfd-8281-33cfb8e991cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:38:42.999430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bec2ed-447c-4dfd-8281-33cfb8e991cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:38:43.000060 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bec2ed-447c-4dfd-8281-33cfb8e991cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:38:43.001626 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bec2ed-447c-4dfd-8281-33cfb8e991cd admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:38:43.011683 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1352/2703] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:38:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:38:43.016163 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763ef198-f64a-4797-b96c-43c4722c9c5a admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:38:43.025872 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763ef198-f64a-4797-b96c-43c4722c9c5a admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:38:43.030411 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb1efff-0977-47fd-ae6d-9c8ce6aa3f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:38:43.031026 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb1efff-0977-47fd-ae6d-9c8ce6aa3f81 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:38:43.031302 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb1efff-0977-47fd-ae6d-9c8ce6aa3f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:38:43.031840 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb1efff-0977-47fd-ae6d-9c8ce6aa3f81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:38:43.032471 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb1efff-0977-47fd-ae6d-9c8ce6aa3f81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:38:43.034067 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb1efff-0977-47fd-ae6d-9c8ce6aa3f81 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:38:43.044772 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1352/2704] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:38:43.048464 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bec2ed-447c-4dfd-8281-33cfb8e991cd admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:38:43.058351 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bec2ed-447c-4dfd-8281-33cfb8e991cd admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:38:43.063447 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548db45-ae33-442a-ba3a-8dbfffe86105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:38:43.064076 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548db45-ae33-442a-ba3a-8dbfffe86105 admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:38:43.064353 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548db45-ae33-442a-ba3a-8dbfffe86105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:38:43.064623 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548db45-ae33-442a-ba3a-8dbfffe86105 admin admin] PATH_INFO: `/v3/projects/cfc1e3a42706435a93301d76ee7b74ed/users/f5d7b40e78a049cd83d079291c3601ff/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:38:43.080160 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1548db45-ae33-442a-ba3a-8dbfffe86105 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfc1e3a42706435a93301d76ee7b74ed, user_id=f5d7b40e78a049cd83d079291c3601ff, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:38:43.082752 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1548db45-ae33-442a-ba3a-8dbfffe86105 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:38:43.095363 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1353/2705] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:38:43 2022] PUT /identity/v3/projects/cfc1e3a42706435a93301d76ee7b74ed/users/f5d7b40e78a049cd83d079291c3601ff/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:38:43.100325 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566fb1a-7b21-4bf6-8819-9c846916e2de None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:38:43.100613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566fb1a-7b21-4bf6-8819-9c846916e2de None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:38:43.100886 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566fb1a-7b21-4bf6-8819-9c846916e2de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:38:43.120945 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-0566fb1a-7b21-4bf6-8819-9c846916e2de None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:38:43.122134 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-0566fb1a-7b21-4bf6-8819-9c846916e2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:38:43.179844 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1353/2706] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:38:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:38:43.189916 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548db45-ae33-442a-ba3a-8dbfffe86105 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:38:43.191710 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-1548db45-ae33-442a-ba3a-8dbfffe86105 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:38:43.246977 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548db45-ae33-442a-ba3a-8dbfffe86105 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:38:43.251209 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc725e22-b8e0-4e2b-9411-8eaf6b24979a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:38:43.251845 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc725e22-b8e0-4e2b-9411-8eaf6b24979a service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:38:43.252120 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc725e22-b8e0-4e2b-9411-8eaf6b24979a service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:38:43.252388 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc725e22-b8e0-4e2b-9411-8eaf6b24979a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:38:43.261274 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc725e22-b8e0-4e2b-9411-8eaf6b24979a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:38:43.262706 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc725e22-b8e0-4e2b-9411-8eaf6b24979a service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:38:43.272646 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1354/2707] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:38:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:38:47.265817 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c7fac-c9b0-4b91-a9d5-aa8d6d6e622e None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:38:47.265817 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c7fac-c9b0-4b91-a9d5-aa8d6d6e622e None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:38:47.265817 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c7fac-c9b0-4b91-a9d5-aa8d6d6e622e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:38:47.303793 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-d14c7fac-c9b0-4b91-a9d5-aa8d6d6e622e None None] MFA Rules not processed for user `f5d7b40e78a049cd83d079291c3601ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:38:47.305003 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-d14c7fac-c9b0-4b91-a9d5-aa8d6d6e622e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:38:47.363283 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:38:47.376712 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1354/2708] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:38:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:38:47.391348 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc725e22-b8e0-4e2b-9411-8eaf6b24979a service neutron] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:38:47.392814 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-cc725e22-b8e0-4e2b-9411-8eaf6b24979a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:38:47.469369 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc725e22-b8e0-4e2b-9411-8eaf6b24979a service neutron] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:38:47.474881 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49832fa2-02fd-470e-8b39-e4e3497f45a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:38:47.475461 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49832fa2-02fd-470e-8b39-e4e3497f45a5 service nova] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:38:47.475742 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49832fa2-02fd-470e-8b39-e4e3497f45a5 service nova] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:38:47.476010 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49832fa2-02fd-470e-8b39-e4e3497f45a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:38:47.482190 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:38:47.486095 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49832fa2-02fd-470e-8b39-e4e3497f45a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:38:47.487962 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49832fa2-02fd-470e-8b39-e4e3497f45a5 service nova] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:38:47.499376 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1355/2709] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:38:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:38:49.090941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58688719-bfb1-4728-85fd-be71b8b5c1ac None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:38:49.091456 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58688719-bfb1-4728-85fd-be71b8b5c1ac None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:38:49.091561 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58688719-bfb1-4728-85fd-be71b8b5c1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:38:49.111669 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-58688719-bfb1-4728-85fd-be71b8b5c1ac None None] MFA Rules not processed for user `f5d7b40e78a049cd83d079291c3601ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:38:49.112869 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-58688719-bfb1-4728-85fd-be71b8b5c1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:38:49.119641 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1355/2710] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:38:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:38:49.429030 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49832fa2-02fd-470e-8b39-e4e3497f45a5 service nova] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:38:49.438324 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49832fa2-02fd-470e-8b39-e4e3497f45a5 service nova] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:38:49.441885 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a269ce-200c-44b6-8ae6-ab72dcee87ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:38:49.442471 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a269ce-200c-44b6-8ae6-ab72dcee87ec service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:38:49.442972 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a269ce-200c-44b6-8ae6-ab72dcee87ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:38:49.443242 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a269ce-200c-44b6-8ae6-ab72dcee87ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:38:49.444952 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-40a269ce-200c-44b6-8ae6-ab72dcee87ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:38:49.454962 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a269ce-200c-44b6-8ae6-ab72dcee87ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:38:49.456370 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a269ce-200c-44b6-8ae6-ab72dcee87ec service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:38:49.464294 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1356/2711] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:38:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:38:59.105301 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=6 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:38:59.460701 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=8 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:39:02.728014 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58688719-bfb1-4728-85fd-be71b8b5c1ac None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:39:02.741677 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58688719-bfb1-4728-85fd-be71b8b5c1ac None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:39:02.747237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f9b9e7-6c48-4ffe-8dce-1122c2a5e21b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:39:02.747977 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9b9e7-6c48-4ffe-8dce-1122c2a5e21b service nova] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:39:02.748263 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9b9e7-6c48-4ffe-8dce-1122c2a5e21b service nova] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:39:02.748560 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9b9e7-6c48-4ffe-8dce-1122c2a5e21b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:39:02.750782 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-17f9b9e7-6c48-4ffe-8dce-1122c2a5e21b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:39:02.763389 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f9b9e7-6c48-4ffe-8dce-1122c2a5e21b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:39:02.764978 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f9b9e7-6c48-4ffe-8dce-1122c2a5e21b service nova] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:39:02.776281 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1356/2712] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:39:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:39:12.771577 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=6 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:39:22.540681 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9798e07-1ed8-466f-b4c5-a7036c05eebc None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:39:22.541234 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9798e07-1ed8-466f-b4c5-a7036c05eebc None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:39:22.541234 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9798e07-1ed8-466f-b4c5-a7036c05eebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:39:22.560644 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-c9798e07-1ed8-466f-b4c5-a7036c05eebc None None] MFA Rules not processed for user `abb529f39710407abf9d6e1cedd9a603`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:39:22.561856 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-c9798e07-1ed8-466f-b4c5-a7036c05eebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:39:22.617879 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1357/2713] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:39:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:39:23.085742 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f9b9e7-6c48-4ffe-8dce-1122c2a5e21b service nova] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:39:23.087383 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-17f9b9e7-6c48-4ffe-8dce-1122c2a5e21b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:39:23.098789 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f9b9e7-6c48-4ffe-8dce-1122c2a5e21b service nova] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:39:23.102338 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dae05d-df9f-498b-8908-68c24e5c8ad6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:39:23.103001 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dae05d-df9f-498b-8908-68c24e5c8ad6 service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:39:23.103725 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dae05d-df9f-498b-8908-68c24e5c8ad6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:39:23.103998 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dae05d-df9f-498b-8908-68c24e5c8ad6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:39:23.105688 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-e0dae05d-df9f-498b-8908-68c24e5c8ad6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:39:23.115357 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dae05d-df9f-498b-8908-68c24e5c8ad6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:39:23.116783 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dae05d-df9f-498b-8908-68c24e5c8ad6 service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:39:23.125244 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1357/2714] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:39:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:39:32.611210 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=25 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:39:33.122098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=6 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:39:56.335362 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9798e07-1ed8-466f-b4c5-a7036c05eebc None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:39:56.336788 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-c9798e07-1ed8-466f-b4c5-a7036c05eebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:39:56.415947 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9798e07-1ed8-466f-b4c5-a7036c05eebc None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:39:56.422566 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f206111-59e4-4540-b73d-6565bf4f1392 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18fe384ecd454fb298dcc788f41dd44b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:39:56.423234 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f206111-59e4-4540-b73d-6565bf4f1392 service placement] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:39:56.423828 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f206111-59e4-4540-b73d-6565bf4f1392 service placement] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:39:56.423828 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f206111-59e4-4540-b73d-6565bf4f1392 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:39:56.425925 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-7f206111-59e4-4540-b73d-6565bf4f1392 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:39:56.438082 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f206111-59e4-4540-b73d-6565bf4f1392 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:39:56.439716 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f206111-59e4-4540-b73d-6565bf4f1392 service placement] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:39:56.450717 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1358/2715] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:39:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:39:57.755662 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be25ff-8f94-4738-b4e1-468b4205e3b0 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:39:57.756153 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be25ff-8f94-4738-b4e1-468b4205e3b0 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:39:57.756262 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be25ff-8f94-4738-b4e1-468b4205e3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:39:57.775702 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-b8be25ff-8f94-4738-b4e1-468b4205e3b0 None None] MFA Rules not processed for user `ffe9a464991e4965a0638d0c23b17ae7`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:39:57.776893 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-b8be25ff-8f94-4738-b4e1-468b4205e3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:39:57.832096 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1358/2716] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:39:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:40:06.446004 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=32 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:40:07.824685 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=25 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:40:50.179956 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f206111-59e4-4540-b73d-6565bf4f1392 service placement] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:40:50.181422 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-7f206111-59e4-4540-b73d-6565bf4f1392 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:40:50.196769 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f206111-59e4-4540-b73d-6565bf4f1392 service placement] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:40:50.202003 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d33abc-a4b3-4812-b630-9543fc35d688 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:40:50.202613 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d33abc-a4b3-4812-b630-9543fc35d688 service nova] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:40:50.202921 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d33abc-a4b3-4812-b630-9543fc35d688 service nova] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:40:50.203192 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d33abc-a4b3-4812-b630-9543fc35d688 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:40:50.204804 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-24d33abc-a4b3-4812-b630-9543fc35d688 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:40:50.216325 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d33abc-a4b3-4812-b630-9543fc35d688 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:40:50.217900 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d33abc-a4b3-4812-b630-9543fc35d688 service nova] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:40:50.227060 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1359/2717] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:40:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:41:00.222916 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=6 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:41:07.614342 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de41f4-8644-4b3d-acc8-77145416d05e None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:41:07.614890 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de41f4-8644-4b3d-acc8-77145416d05e None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:41:07.614998 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de41f4-8644-4b3d-acc8-77145416d05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:41:07.633998 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-55de41f4-8644-4b3d-acc8-77145416d05e None None] MFA Rules not processed for user `abb529f39710407abf9d6e1cedd9a603`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:41:07.635219 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-55de41f4-8644-4b3d-acc8-77145416d05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:41:07.641853 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1359/2718] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:41:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:41:16.324250 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d33abc-a4b3-4812-b630-9543fc35d688 service nova] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:41:16.334057 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d33abc-a4b3-4812-b630-9543fc35d688 service nova] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:41:16.337580 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0ee9d-364b-4178-9a5f-083ff25815f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:41:16.338161 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0ee9d-364b-4178-9a5f-083ff25815f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:41:16.338434 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0ee9d-364b-4178-9a5f-083ff25815f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:41:16.338701 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0ee9d-364b-4178-9a5f-083ff25815f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:41:16.340394 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-3ba0ee9d-364b-4178-9a5f-083ff25815f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:41:16.353943 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba0ee9d-364b-4178-9a5f-083ff25815f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:41:16.355432 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba0ee9d-364b-4178-9a5f-083ff25815f0 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:41:16.365962 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1360/2719] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:41:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:41:17.628659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=3 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:41:18.720898 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de41f4-8644-4b3d-acc8-77145416d05e None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:41:18.731439 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de41f4-8644-4b3d-acc8-77145416d05e None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:41:18.734921 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd9156-2445-4a94-b09e-e25f19d3b7ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:41:18.735543 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd9156-2445-4a94-b09e-e25f19d3b7ea service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:41:18.735817 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd9156-2445-4a94-b09e-e25f19d3b7ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:41:18.736082 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd9156-2445-4a94-b09e-e25f19d3b7ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:41:18.737870 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-debd9156-2445-4a94-b09e-e25f19d3b7ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:41:18.750786 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debd9156-2445-4a94-b09e-e25f19d3b7ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:41:18.752229 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debd9156-2445-4a94-b09e-e25f19d3b7ea service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:41:18.762856 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1360/2720] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:41:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:41:26.361373 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=6 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:41:28.758099 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=6 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:42:01.118360 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0ee9d-364b-4178-9a5f-083ff25815f0 service neutron] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:42:01.127456 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0ee9d-364b-4178-9a5f-083ff25815f0 service neutron] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:42:01.131471 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb578a6-4779-4328-8c47-b024385e231d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:42:01.132119 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb578a6-4779-4328-8c47-b024385e231d service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:42:01.132395 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb578a6-4779-4328-8c47-b024385e231d service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:42:01.132667 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb578a6-4779-4328-8c47-b024385e231d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:42:01.134473 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-4eb578a6-4779-4328-8c47-b024385e231d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:42:01.144834 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb578a6-4779-4328-8c47-b024385e231d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:42:01.146269 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb578a6-4779-4328-8c47-b024385e231d service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:42:01.154719 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1361/2721] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:42:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:42:07.231375 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd9156-2445-4a94-b09e-e25f19d3b7ea service neutron] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:42:07.232780 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-debd9156-2445-4a94-b09e-e25f19d3b7ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:42:07.308485 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd9156-2445-4a94-b09e-e25f19d3b7ea service neutron] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:42:07.311995 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334f101-6c2a-4cbb-8298-6900ea4b1783 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:42:07.312583 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334f101-6c2a-4cbb-8298-6900ea4b1783 service cinder] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:42:07.312857 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334f101-6c2a-4cbb-8298-6900ea4b1783 service cinder] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:42:07.313123 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334f101-6c2a-4cbb-8298-6900ea4b1783 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:42:07.314800 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-9334f101-6c2a-4cbb-8298-6900ea4b1783 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:42:07.325064 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9334f101-6c2a-4cbb-8298-6900ea4b1783 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:42:07.326481 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9334f101-6c2a-4cbb-8298-6900ea4b1783 service cinder] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:42:07.335706 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1361/2722] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:42:07 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:42:11.150836 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=6 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:42:17.331723 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=35 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:42:18.349338 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb34b35-47a8-40de-a8ac-71d5afe22c3e None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:42:18.349858 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb34b35-47a8-40de-a8ac-71d5afe22c3e None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:42:18.349858 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb34b35-47a8-40de-a8ac-71d5afe22c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:42:18.369338 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-1cb34b35-47a8-40de-a8ac-71d5afe22c3e None None] MFA Rules not processed for user `ffe9a464991e4965a0638d0c23b17ae7`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:42:18.370523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-1cb34b35-47a8-40de-a8ac-71d5afe22c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:42:18.377266 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1362/2723] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:42:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:42:19.133675 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334f101-6c2a-4cbb-8298-6900ea4b1783 service cinder] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:42:19.155127 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334f101-6c2a-4cbb-8298-6900ea4b1783 service cinder] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:42:19.160278 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d360c75-d29c-4c0d-892a-445b8b0cbde8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18fe384ecd454fb298dcc788f41dd44b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:42:19.160885 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d360c75-d29c-4c0d-892a-445b8b0cbde8 service placement] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:42:19.161413 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d360c75-d29c-4c0d-892a-445b8b0cbde8 service placement] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:42:19.161681 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d360c75-d29c-4c0d-892a-445b8b0cbde8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:42:19.163454 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-5d360c75-d29c-4c0d-892a-445b8b0cbde8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:42:19.175180 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d360c75-d29c-4c0d-892a-445b8b0cbde8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:42:19.177340 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d360c75-d29c-4c0d-892a-445b8b0cbde8 service placement] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:42:19.187502 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1362/2724] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:42:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:42:21.772527 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb34b35-47a8-40de-a8ac-71d5afe22c3e None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:42:21.787537 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb34b35-47a8-40de-a8ac-71d5afe22c3e None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:42:21.793528 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5ade6-e79b-4043-bfab-33483ae63ca1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:42:21.794130 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5ade6-e79b-4043-bfab-33483ae63ca1 service nova] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:42:21.794578 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5ade6-e79b-4043-bfab-33483ae63ca1 service nova] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:42:21.794919 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5ade6-e79b-4043-bfab-33483ae63ca1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:42:21.796718 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-7fa5ade6-e79b-4043-bfab-33483ae63ca1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:42:21.813657 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5ade6-e79b-4043-bfab-33483ae63ca1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:42:21.819994 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5ade6-e79b-4043-bfab-33483ae63ca1 service nova] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:42:21.843608 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1363/2725] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:42:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:42:29.183203 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=6 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:42:31.825326 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=9 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:42:33.758076 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d360c75-d29c-4c0d-892a-445b8b0cbde8 service placement] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:42:33.771521 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d360c75-d29c-4c0d-892a-445b8b0cbde8 service placement] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:42:33.777217 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3b11cc-8132-4988-8f4c-d2206e3ba601 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:42:33.777822 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b11cc-8132-4988-8f4c-d2206e3ba601 service nova] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:42:33.778103 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b11cc-8132-4988-8f4c-d2206e3ba601 service nova] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:42:33.778379 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b11cc-8132-4988-8f4c-d2206e3ba601 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:42:33.780123 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-ff3b11cc-8132-4988-8f4c-d2206e3ba601 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:42:33.793341 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3b11cc-8132-4988-8f4c-d2206e3ba601 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:42:33.794994 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3b11cc-8132-4988-8f4c-d2206e3ba601 service nova] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:42:33.806321 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1363/2726] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:42:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:42:37.215509 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5ade6-e79b-4043-bfab-33483ae63ca1 service nova] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:42:37.224760 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5ade6-e79b-4043-bfab-33483ae63ca1 service nova] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:42:37.228270 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c83e5b-7219-47d2-9ccd-a9ea765a00e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:42:37.228856 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c83e5b-7219-47d2-9ccd-a9ea765a00e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:42:37.229131 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c83e5b-7219-47d2-9ccd-a9ea765a00e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:42:37.229404 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c83e5b-7219-47d2-9ccd-a9ea765a00e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:42:37.231139 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-53c83e5b-7219-47d2-9ccd-a9ea765a00e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:42:37.291998 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c83e5b-7219-47d2-9ccd-a9ea765a00e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:42:37.293408 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c83e5b-7219-47d2-9ccd-a9ea765a00e6 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:42:37.302639 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1364/2727] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:42:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:42:43.801093 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=6 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:42:47.298491 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=30 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:43:05.772320 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3b11cc-8132-4988-8f4c-d2206e3ba601 service nova] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:05.781542 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3b11cc-8132-4988-8f4c-d2206e3ba601 service nova] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:05.785045 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55963c27-361b-48bf-8665-b1629021287b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:05.785633 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55963c27-361b-48bf-8665-b1629021287b service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:05.785909 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55963c27-361b-48bf-8665-b1629021287b service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:05.786179 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55963c27-361b-48bf-8665-b1629021287b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:05.788637 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-55963c27-361b-48bf-8665-b1629021287b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:43:05.800110 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55963c27-361b-48bf-8665-b1629021287b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:05.801529 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55963c27-361b-48bf-8665-b1629021287b service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:05.811406 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1364/2728] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:43:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:43:08.883843 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c83e5b-7219-47d2-9ccd-a9ea765a00e6 service neutron] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:08.885876 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-53c83e5b-7219-47d2-9ccd-a9ea765a00e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:43:08.904972 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c83e5b-7219-47d2-9ccd-a9ea765a00e6 service neutron] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:08.909964 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652b0337-cd46-4022-bc64-afa6735f794c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:08.910608 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b0337-cd46-4022-bc64-afa6735f794c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:08.910922 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b0337-cd46-4022-bc64-afa6735f794c admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:08.911201 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b0337-cd46-4022-bc64-afa6735f794c admin admin] PATH_INFO: `/v3/users/abb529f39710407abf9d6e1cedd9a603` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:08.913167 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652b0337-cd46-4022-bc64-afa6735f794c admin admin] RBAC: Authorizing `identity:delete_user(user_id=abb529f39710407abf9d6e1cedd9a603)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:08.915172 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652b0337-cd46-4022-bc64-afa6735f794c admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:08.970341 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-652b0337-cd46-4022-bc64-afa6735f794c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abb529f39710407abf9d6e1cedd9a603'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:08.971227 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-652b0337-cd46-4022-bc64-afa6735f794c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abb529f39710407abf9d6e1cedd9a603'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:08.971854 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-652b0337-cd46-4022-bc64-afa6735f794c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abb529f39710407abf9d6e1cedd9a603'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:08.986317 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-652b0337-cd46-4022-bc64-afa6735f794c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abb529f39710407abf9d6e1cedd9a603'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:08.994104 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-652b0337-cd46-4022-bc64-afa6735f794c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abb529f39710407abf9d6e1cedd9a603'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:09.010574 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1365/2729] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:43:08 2022] DELETE /identity/v3/users/abb529f39710407abf9d6e1cedd9a603 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:43:09.187053 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55963c27-361b-48bf-8665-b1629021287b service neutron] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:09.197923 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55963c27-361b-48bf-8665-b1629021287b service neutron] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:09.255553 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639d0ca-8cad-47cb-88dd-4c0f088a8627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:09.256849 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639d0ca-8cad-47cb-88dd-4c0f088a8627 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:09.257131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639d0ca-8cad-47cb-88dd-4c0f088a8627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:09.257405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639d0ca-8cad-47cb-88dd-4c0f088a8627 admin admin] PATH_INFO: `/v3/projects/21710ac430424ec9bf33aabfda5eb8ba` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:09.258574 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d639d0ca-8cad-47cb-88dd-4c0f088a8627 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21710ac430424ec9bf33aabfda5eb8ba)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:09.260561 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d639d0ca-8cad-47cb-88dd-4c0f088a8627 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:09.305750 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-d639d0ca-8cad-47cb-88dd-4c0f088a8627 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21710ac430424ec9bf33aabfda5eb8ba', 'request_id': 'req-d639d0ca-8cad-47cb-88dd-4c0f088a8627'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:09.311932 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-d639d0ca-8cad-47cb-88dd-4c0f088a8627 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21710ac430424ec9bf33aabfda5eb8ba', 'request_id': 'req-d639d0ca-8cad-47cb-88dd-4c0f088a8627'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:09.317403 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-d639d0ca-8cad-47cb-88dd-4c0f088a8627 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21710ac430424ec9bf33aabfda5eb8ba', 'request_id': 'req-d639d0ca-8cad-47cb-88dd-4c0f088a8627'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:09.329865 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-d639d0ca-8cad-47cb-88dd-4c0f088a8627 admin admin] The token cache is being invalidate because project 21710ac430424ec9bf33aabfda5eb8ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:43:09.330156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-d639d0ca-8cad-47cb-88dd-4c0f088a8627 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:09.330769 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-d639d0ca-8cad-47cb-88dd-4c0f088a8627 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:09.332299 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1365/2730] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:43:09 2022] DELETE /identity/v3/projects/21710ac430424ec9bf33aabfda5eb8ba => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:43:09.337204 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65776007-bac8-4b21-87a0-053e2b32678c None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:09.337492 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65776007-bac8-4b21-87a0-053e2b32678c None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:09.337770 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65776007-bac8-4b21-87a0-053e2b32678c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:09.356861 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-65776007-bac8-4b21-87a0-053e2b32678c None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:43:09.358059 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-65776007-bac8-4b21-87a0-053e2b32678c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:43:09.417185 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1366/2731] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:43:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:43:09.424933 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6bed1-5307-48f0-b783-4e1e221ae482 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:09.425219 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6bed1-5307-48f0-b783-4e1e221ae482 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:09.425495 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6bed1-5307-48f0-b783-4e1e221ae482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:09.446112 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-05e6bed1-5307-48f0-b783-4e1e221ae482 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:43:09.447333 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-05e6bed1-5307-48f0-b783-4e1e221ae482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:43:09.455031 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1366/2732] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:43:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:43:09.459625 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65776007-bac8-4b21-87a0-053e2b32678c None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:09.469129 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65776007-bac8-4b21-87a0-053e2b32678c None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:09.473608 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eadf96-ac86-4ae6-8458-9473a9c2f29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:09.474194 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eadf96-ac86-4ae6-8458-9473a9c2f29b admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:09.474494 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eadf96-ac86-4ae6-8458-9473a9c2f29b admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:09.474780 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eadf96-ac86-4ae6-8458-9473a9c2f29b admin admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:09.475428 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eadf96-ac86-4ae6-8458-9473a9c2f29b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:43:09.475726 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eadf96-ac86-4ae6-8458-9473a9c2f29b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:09.477287 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eadf96-ac86-4ae6-8458-9473a9c2f29b admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:09.488447 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1367/2733] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:43:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:43:09.492623 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e6bed1-5307-48f0-b783-4e1e221ae482 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:09.501838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e6bed1-5307-48f0-b783-4e1e221ae482 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:09.506991 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8b9de-893c-4063-816d-63334ea07253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:09.507583 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8b9de-893c-4063-816d-63334ea07253 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:09.507861 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8b9de-893c-4063-816d-63334ea07253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:09.508131 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8b9de-893c-4063-816d-63334ea07253 admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:09.508760 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad8b9de-893c-4063-816d-63334ea07253 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:09.510648 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad8b9de-893c-4063-816d-63334ea07253 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:09.537884 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1367/2734] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:43:09 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:43:09.541941 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eadf96-ac86-4ae6-8458-9473a9c2f29b admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:09.552425 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eadf96-ac86-4ae6-8458-9473a9c2f29b admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:09.609227 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc18228-acbb-4828-a890-53359703d455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:09.609745 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc18228-acbb-4828-a890-53359703d455 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:09.610021 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc18228-acbb-4828-a890-53359703d455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:09.610292 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc18228-acbb-4828-a890-53359703d455 admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:09.610962 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc18228-acbb-4828-a890-53359703d455 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:09.613153 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc18228-acbb-4828-a890-53359703d455 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:09.621306 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:43:09.636446 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1368/2735] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:43:09 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:43:09.640599 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8b9de-893c-4063-816d-63334ea07253 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:09.651465 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8b9de-893c-4063-816d-63334ea07253 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:09.656033 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf5a27-4e7c-4dc8-8ded-9d27bdd97a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:09.656637 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf5a27-4e7c-4dc8-8ded-9d27bdd97a2d admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:09.656919 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf5a27-4e7c-4dc8-8ded-9d27bdd97a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:09.657196 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf5a27-4e7c-4dc8-8ded-9d27bdd97a2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:09.657849 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaf5a27-4e7c-4dc8-8ded-9d27bdd97a2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:09.659679 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaf5a27-4e7c-4dc8-8ded-9d27bdd97a2d admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:09.671588 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1368/2736] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:43:09.676527 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc18228-acbb-4828-a890-53359703d455 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:09.687220 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc18228-acbb-4828-a890-53359703d455 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:09.692079 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f934e7-787e-41fa-afcc-3ba53418a690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:09.692709 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f934e7-787e-41fa-afcc-3ba53418a690 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:09.693020 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f934e7-787e-41fa-afcc-3ba53418a690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:09.693764 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f934e7-787e-41fa-afcc-3ba53418a690 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:09.694421 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f934e7-787e-41fa-afcc-3ba53418a690 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:09.696050 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f934e7-787e-41fa-afcc-3ba53418a690 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:09.707984 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1369/2737] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:43:09.712704 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf5a27-4e7c-4dc8-8ded-9d27bdd97a2d admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:09.722430 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf5a27-4e7c-4dc8-8ded-9d27bdd97a2d admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:09.726966 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0151c77-1cb6-4d16-aa94-ddbb24e1a47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:09.727604 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0151c77-1cb6-4d16-aa94-ddbb24e1a47c admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:09.727876 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0151c77-1cb6-4d16-aa94-ddbb24e1a47c admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:09.728144 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0151c77-1cb6-4d16-aa94-ddbb24e1a47c admin admin] PATH_INFO: `/v3/projects/5fbbe04a994241eda6148e160b5c04aa/users/30dd9b46eba548599c04950e26de62c9/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:09.742705 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0151c77-1cb6-4d16-aa94-ddbb24e1a47c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fbbe04a994241eda6148e160b5c04aa, user_id=30dd9b46eba548599c04950e26de62c9, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:09.745352 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0151c77-1cb6-4d16-aa94-ddbb24e1a47c admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:09.758865 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1369/2738] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:43:09 2022] PUT /identity/v3/projects/5fbbe04a994241eda6148e160b5c04aa/users/30dd9b46eba548599c04950e26de62c9/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:43:09.769207 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999838cb-4f7b-4a78-bb2c-2ae45b4cab82 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:09.769537 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999838cb-4f7b-4a78-bb2c-2ae45b4cab82 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:09.769807 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999838cb-4f7b-4a78-bb2c-2ae45b4cab82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:09.796579 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-999838cb-4f7b-4a78-bb2c-2ae45b4cab82 None None] MFA Rules not processed for user `30dd9b46eba548599c04950e26de62c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:43:09.797765 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-999838cb-4f7b-4a78-bb2c-2ae45b4cab82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:43:09.860658 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1370/2739] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:43:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:43:09.865538 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0151c77-1cb6-4d16-aa94-ddbb24e1a47c admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:09.878726 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:43:09.931446 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0151c77-1cb6-4d16-aa94-ddbb24e1a47c admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:09.936265 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5118c443-7978-430a-b85e-a87a6b980a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:09.936863 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118c443-7978-430a-b85e-a87a6b980a89 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:09.937256 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118c443-7978-430a-b85e-a87a6b980a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:09.937552 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118c443-7978-430a-b85e-a87a6b980a89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:09.938184 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5118c443-7978-430a-b85e-a87a6b980a89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:09.940100 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5118c443-7978-430a-b85e-a87a6b980a89 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:09.965940 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1370/2740] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:43:09 2022] POST /identity/v3/projects => generated 371 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:43:09.970367 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999838cb-4f7b-4a78-bb2c-2ae45b4cab82 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:09.980846 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999838cb-4f7b-4a78-bb2c-2ae45b4cab82 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:10.035533 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd6d19-9e55-4b99-bc70-b6ba1f70a6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:10.036128 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd6d19-9e55-4b99-bc70-b6ba1f70a6a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:10.036406 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd6d19-9e55-4b99-bc70-b6ba1f70a6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:10.036937 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd6d19-9e55-4b99-bc70-b6ba1f70a6a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:10.037579 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cd6d19-9e55-4b99-bc70-b6ba1f70a6a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:10.039553 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cd6d19-9e55-4b99-bc70-b6ba1f70a6a2 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:10.061551 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1371/2741] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:43:09 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:43:10.065506 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5118c443-7978-430a-b85e-a87a6b980a89 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:10.076253 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5118c443-7978-430a-b85e-a87a6b980a89 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:10.080732 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320435b7-90f5-4edc-88cf-8be9984252f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:10.081321 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320435b7-90f5-4edc-88cf-8be9984252f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:10.081595 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320435b7-90f5-4edc-88cf-8be9984252f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:10.081864 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320435b7-90f5-4edc-88cf-8be9984252f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:10.082494 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320435b7-90f5-4edc-88cf-8be9984252f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:10.084114 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320435b7-90f5-4edc-88cf-8be9984252f6 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:10.094384 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1371/2742] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:43:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:43:10.098616 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd6d19-9e55-4b99-bc70-b6ba1f70a6a2 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:10.108535 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd6d19-9e55-4b99-bc70-b6ba1f70a6a2 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:10.113121 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a332ad8e-be3e-45c5-a7fa-04155aeae42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:10.113705 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a332ad8e-be3e-45c5-a7fa-04155aeae42b admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:10.113985 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a332ad8e-be3e-45c5-a7fa-04155aeae42b admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:10.114257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a332ad8e-be3e-45c5-a7fa-04155aeae42b admin admin] PATH_INFO: `/v3/projects/0ed1492f699548cb939d4049c7d6313b/users/09ddfaae12804890ad6fd4d6a07399a0/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:10.128975 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a332ad8e-be3e-45c5-a7fa-04155aeae42b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ed1492f699548cb939d4049c7d6313b, user_id=09ddfaae12804890ad6fd4d6a07399a0, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:10.133290 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a332ad8e-be3e-45c5-a7fa-04155aeae42b admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:10.145065 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1372/2743] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:43:10 2022] PUT /identity/v3/projects/0ed1492f699548cb939d4049c7d6313b/users/09ddfaae12804890ad6fd4d6a07399a0/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:43:10.149053 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320435b7-90f5-4edc-88cf-8be9984252f6 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:10.158938 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320435b7-90f5-4edc-88cf-8be9984252f6 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:10.213489 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166cbc86-bb6f-42c2-ab2e-86e92e2e655e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:10.214125 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166cbc86-bb6f-42c2-ab2e-86e92e2e655e admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:10.214343 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166cbc86-bb6f-42c2-ab2e-86e92e2e655e admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:10.214613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166cbc86-bb6f-42c2-ab2e-86e92e2e655e admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:10.215274 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166cbc86-bb6f-42c2-ab2e-86e92e2e655e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:10.216851 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166cbc86-bb6f-42c2-ab2e-86e92e2e655e admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:10.227189 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1372/2744] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:43:10 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:43:10.231218 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a332ad8e-be3e-45c5-a7fa-04155aeae42b admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:10.241965 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a332ad8e-be3e-45c5-a7fa-04155aeae42b admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:10.246573 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48890bf8-ff17-4a29-aa1d-438f463953f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:10.247252 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48890bf8-ff17-4a29-aa1d-438f463953f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:10.247532 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48890bf8-ff17-4a29-aa1d-438f463953f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:10.247806 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48890bf8-ff17-4a29-aa1d-438f463953f3 admin admin] PATH_INFO: `/v3/projects/0ed1492f699548cb939d4049c7d6313b/users/09ddfaae12804890ad6fd4d6a07399a0/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:10.249710 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48890bf8-ff17-4a29-aa1d-438f463953f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ed1492f699548cb939d4049c7d6313b, user_id=09ddfaae12804890ad6fd4d6a07399a0, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:10.252352 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48890bf8-ff17-4a29-aa1d-438f463953f3 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:10.262256 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1373/2745] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:43:10 2022] PUT /identity/v3/projects/0ed1492f699548cb939d4049c7d6313b/users/09ddfaae12804890ad6fd4d6a07399a0/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:43:10.273603 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9097921-e6fd-4706-b1bb-0231bd315cd6 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:10.273889 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9097921-e6fd-4706-b1bb-0231bd315cd6 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:10.274162 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9097921-e6fd-4706-b1bb-0231bd315cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:10.303368 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-b9097921-e6fd-4706-b1bb-0231bd315cd6 None None] MFA Rules not processed for user `09ddfaae12804890ad6fd4d6a07399a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:43:10.304568 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-b9097921-e6fd-4706-b1bb-0231bd315cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:43:10.370874 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1373/2746] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:43:10 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:43:10.381795 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48890bf8-ff17-4a29-aa1d-438f463953f3 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:10.383630 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-48890bf8-ff17-4a29-aa1d-438f463953f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:43:10.405510 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:43:10.441571 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48890bf8-ff17-4a29-aa1d-438f463953f3 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:10.445221 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0cacf7-e4db-43c7-ae01-fe71a19abe47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:10.445809 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0cacf7-e4db-43c7-ae01-fe71a19abe47 service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:10.446085 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0cacf7-e4db-43c7-ae01-fe71a19abe47 service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:10.446355 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0cacf7-e4db-43c7-ae01-fe71a19abe47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:10.455353 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0cacf7-e4db-43c7-ae01-fe71a19abe47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:10.456778 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0cacf7-e4db-43c7-ae01-fe71a19abe47 service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:10.466239 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1374/2747] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:43:10 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:43:10.852533 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9097921-e6fd-4706-b1bb-0231bd315cd6 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:10.862186 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:43:10.919244 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9097921-e6fd-4706-b1bb-0231bd315cd6 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:10.924073 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea867d-053b-4eca-8b97-6fea6ec3252f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:10.924696 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea867d-053b-4eca-8b97-6fea6ec3252f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:10.924974 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea867d-053b-4eca-8b97-6fea6ec3252f admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:10.925242 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea867d-053b-4eca-8b97-6fea6ec3252f admin admin] PATH_INFO: `/v3/users/30dd9b46eba548599c04950e26de62c9` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:10.926458 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ea867d-053b-4eca-8b97-6fea6ec3252f admin admin] RBAC: Authorizing `identity:delete_user(user_id=30dd9b46eba548599c04950e26de62c9)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:10.928427 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ea867d-053b-4eca-8b97-6fea6ec3252f admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:10.973153 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-00ea867d-053b-4eca-8b97-6fea6ec3252f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30dd9b46eba548599c04950e26de62c9'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:10.973790 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-00ea867d-053b-4eca-8b97-6fea6ec3252f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30dd9b46eba548599c04950e26de62c9'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:10.980706 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-00ea867d-053b-4eca-8b97-6fea6ec3252f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30dd9b46eba548599c04950e26de62c9'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:10.981368 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-00ea867d-053b-4eca-8b97-6fea6ec3252f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30dd9b46eba548599c04950e26de62c9'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:10.993368 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-00ea867d-053b-4eca-8b97-6fea6ec3252f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30dd9b46eba548599c04950e26de62c9'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:11.005472 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1374/2748] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:43:10 2022] DELETE /identity/v3/users/30dd9b46eba548599c04950e26de62c9 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:43:11.010416 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2505e67-c27b-40ec-800c-d1fa0c710904 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:11.010909 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2505e67-c27b-40ec-800c-d1fa0c710904 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:11.011019 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2505e67-c27b-40ec-800c-d1fa0c710904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:11.030004 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-c2505e67-c27b-40ec-800c-d1fa0c710904 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:43:11.031226 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-c2505e67-c27b-40ec-800c-d1fa0c710904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:43:11.088414 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1375/2749] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:43:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:43:11.097979 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea867d-053b-4eca-8b97-6fea6ec3252f admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:11.099459 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-00ea867d-053b-4eca-8b97-6fea6ec3252f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:43:11.156457 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea867d-053b-4eca-8b97-6fea6ec3252f admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:11.160089 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb91f7ab-06d5-46c6-a3b1-b70dcabd1ea4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:11.160688 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91f7ab-06d5-46c6-a3b1-b70dcabd1ea4 service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:11.160964 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91f7ab-06d5-46c6-a3b1-b70dcabd1ea4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:11.161236 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91f7ab-06d5-46c6-a3b1-b70dcabd1ea4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:11.171524 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb91f7ab-06d5-46c6-a3b1-b70dcabd1ea4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:11.172876 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb91f7ab-06d5-46c6-a3b1-b70dcabd1ea4 service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:11.182906 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1375/2750] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:43:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:43:11.440547 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2505e67-c27b-40ec-800c-d1fa0c710904 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:11.441997 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-c2505e67-c27b-40ec-800c-d1fa0c710904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:43:11.456811 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2505e67-c27b-40ec-800c-d1fa0c710904 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:11.462492 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f11743d-d1ad-4993-8a8a-5107e9c31ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:11.463232 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11743d-d1ad-4993-8a8a-5107e9c31ce3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:11.463425 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11743d-d1ad-4993-8a8a-5107e9c31ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:11.463739 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11743d-d1ad-4993-8a8a-5107e9c31ce3 admin admin] PATH_INFO: `/v3/projects/5fbbe04a994241eda6148e160b5c04aa` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:11.464845 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f11743d-d1ad-4993-8a8a-5107e9c31ce3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fbbe04a994241eda6148e160b5c04aa)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:11.466703 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f11743d-d1ad-4993-8a8a-5107e9c31ce3 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:11.515275 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9f11743d-d1ad-4993-8a8a-5107e9c31ce3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fbbe04a994241eda6148e160b5c04aa', 'request_id': 'req-9f11743d-d1ad-4993-8a8a-5107e9c31ce3'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:11.524306 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9f11743d-d1ad-4993-8a8a-5107e9c31ce3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fbbe04a994241eda6148e160b5c04aa', 'request_id': 'req-9f11743d-d1ad-4993-8a8a-5107e9c31ce3'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:11.532230 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9f11743d-d1ad-4993-8a8a-5107e9c31ce3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fbbe04a994241eda6148e160b5c04aa', 'request_id': 'req-9f11743d-d1ad-4993-8a8a-5107e9c31ce3'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:11.547523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9f11743d-d1ad-4993-8a8a-5107e9c31ce3 admin admin] The token cache is being invalidate because project 5fbbe04a994241eda6148e160b5c04aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:43:11.547813 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9f11743d-d1ad-4993-8a8a-5107e9c31ce3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:11.548464 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9f11743d-d1ad-4993-8a8a-5107e9c31ce3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:11.550399 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1376/2751] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:43:11 2022] DELETE /identity/v3/projects/5fbbe04a994241eda6148e160b5c04aa => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:43:11.555284 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb91f7ab-06d5-46c6-a3b1-b70dcabd1ea4 service neutron] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:11.567156 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb91f7ab-06d5-46c6-a3b1-b70dcabd1ea4 service neutron] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:11.598675 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:43:11.627550 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4c5fd2-10bf-48a8-932b-0169826a7a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:11.628172 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c5fd2-10bf-48a8-932b-0169826a7a14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:11.628450 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c5fd2-10bf-48a8-932b-0169826a7a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:11.628721 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c5fd2-10bf-48a8-932b-0169826a7a14 admin admin] PATH_INFO: `/v3/users/09ddfaae12804890ad6fd4d6a07399a0` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:11.629812 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4c5fd2-10bf-48a8-932b-0169826a7a14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09ddfaae12804890ad6fd4d6a07399a0)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:11.632382 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4c5fd2-10bf-48a8-932b-0169826a7a14 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:11.679824 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-cd4c5fd2-10bf-48a8-932b-0169826a7a14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09ddfaae12804890ad6fd4d6a07399a0'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:11.680468 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-cd4c5fd2-10bf-48a8-932b-0169826a7a14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09ddfaae12804890ad6fd4d6a07399a0'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:11.687355 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-cd4c5fd2-10bf-48a8-932b-0169826a7a14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09ddfaae12804890ad6fd4d6a07399a0'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:11.687949 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-cd4c5fd2-10bf-48a8-932b-0169826a7a14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09ddfaae12804890ad6fd4d6a07399a0'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:11.699684 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-cd4c5fd2-10bf-48a8-932b-0169826a7a14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09ddfaae12804890ad6fd4d6a07399a0'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:11.712856 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1376/2752] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:43:11 2022] DELETE /identity/v3/users/09ddfaae12804890ad6fd4d6a07399a0 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:43:11.980673 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f11743d-d1ad-4993-8a8a-5107e9c31ce3 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:11.982579 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-9f11743d-d1ad-4993-8a8a-5107e9c31ce3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:43:11.996452 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f11743d-d1ad-4993-8a8a-5107e9c31ce3 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:12.040725 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:43:12.054376 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097b19d-dc98-4494-a043-b1f5d3701869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:12.055041 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097b19d-dc98-4494-a043-b1f5d3701869 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:12.055325 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097b19d-dc98-4494-a043-b1f5d3701869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:12.055600 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097b19d-dc98-4494-a043-b1f5d3701869 admin admin] PATH_INFO: `/v3/projects/0ed1492f699548cb939d4049c7d6313b` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:12.056709 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097b19d-dc98-4494-a043-b1f5d3701869 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ed1492f699548cb939d4049c7d6313b)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:12.058620 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097b19d-dc98-4494-a043-b1f5d3701869 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:12.104517 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-3097b19d-dc98-4494-a043-b1f5d3701869 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ed1492f699548cb939d4049c7d6313b', 'request_id': 'req-3097b19d-dc98-4494-a043-b1f5d3701869'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:12.110357 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-3097b19d-dc98-4494-a043-b1f5d3701869 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ed1492f699548cb939d4049c7d6313b', 'request_id': 'req-3097b19d-dc98-4494-a043-b1f5d3701869'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:12.116047 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-3097b19d-dc98-4494-a043-b1f5d3701869 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ed1492f699548cb939d4049c7d6313b', 'request_id': 'req-3097b19d-dc98-4494-a043-b1f5d3701869'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:12.127846 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-3097b19d-dc98-4494-a043-b1f5d3701869 admin admin] The token cache is being invalidate because project 0ed1492f699548cb939d4049c7d6313b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:43:12.128130 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-3097b19d-dc98-4494-a043-b1f5d3701869 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:12.128747 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-3097b19d-dc98-4494-a043-b1f5d3701869 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:12.130228 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1377/2753] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:43:11 2022] DELETE /identity/v3/projects/0ed1492f699548cb939d4049c7d6313b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:43:12.135920 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f77895-7c2c-4161-a1b2-ddf80152912c None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:12.136205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f77895-7c2c-4161-a1b2-ddf80152912c None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:12.136478 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f77895-7c2c-4161-a1b2-ddf80152912c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:12.277662 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-60f77895-7c2c-4161-a1b2-ddf80152912c None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:43:12.278769 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-60f77895-7c2c-4161-a1b2-ddf80152912c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:43:12.337692 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1377/2754] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:43:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:43:12.345333 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f163ec4-a692-4db9-a8d5-1a40726c370b None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:12.345621 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f163ec4-a692-4db9-a8d5-1a40726c370b None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:12.345897 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f163ec4-a692-4db9-a8d5-1a40726c370b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:12.367318 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-2f163ec4-a692-4db9-a8d5-1a40726c370b None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:43:12.368528 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-2f163ec4-a692-4db9-a8d5-1a40726c370b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:43:12.376312 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1378/2755] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:43:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:43:12.380450 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f77895-7c2c-4161-a1b2-ddf80152912c None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:12.390434 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f77895-7c2c-4161-a1b2-ddf80152912c None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:12.394969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4910499b-dc01-4e04-8489-48dcfb2dd50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:12.395571 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910499b-dc01-4e04-8489-48dcfb2dd50f admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:12.395849 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910499b-dc01-4e04-8489-48dcfb2dd50f admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:12.396117 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910499b-dc01-4e04-8489-48dcfb2dd50f admin admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:12.397336 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4910499b-dc01-4e04-8489-48dcfb2dd50f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:43:12.397633 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4910499b-dc01-4e04-8489-48dcfb2dd50f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:12.399315 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4910499b-dc01-4e04-8489-48dcfb2dd50f admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:12.410559 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1378/2756] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:43:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:43:12.414916 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f163ec4-a692-4db9-a8d5-1a40726c370b None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:12.423877 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f163ec4-a692-4db9-a8d5-1a40726c370b None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:12.428392 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630720c5-44ea-423f-b592-6d309647b0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:12.428975 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630720c5-44ea-423f-b592-6d309647b0f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:12.429249 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630720c5-44ea-423f-b592-6d309647b0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:12.429518 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630720c5-44ea-423f-b592-6d309647b0f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:12.430150 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630720c5-44ea-423f-b592-6d309647b0f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:12.432043 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630720c5-44ea-423f-b592-6d309647b0f4 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:12.458344 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1379/2757] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:43:12 2022] POST /identity/v3/projects => generated 381 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:43:12.462991 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4910499b-dc01-4e04-8489-48dcfb2dd50f admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:12.472947 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4910499b-dc01-4e04-8489-48dcfb2dd50f admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:12.504491 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:43:12.530486 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe20fc-3066-48ae-ac95-c30592f52ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:12.531052 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe20fc-3066-48ae-ac95-c30592f52ca4 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:12.531329 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe20fc-3066-48ae-ac95-c30592f52ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:12.531596 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe20fc-3066-48ae-ac95-c30592f52ca4 admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:12.532205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafe20fc-3066-48ae-ac95-c30592f52ca4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:12.534086 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafe20fc-3066-48ae-ac95-c30592f52ca4 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:12.556095 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1379/2758] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:43:12 2022] POST /identity/v3/users => generated 291 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:43:12.560186 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630720c5-44ea-423f-b592-6d309647b0f4 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:12.570789 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630720c5-44ea-423f-b592-6d309647b0f4 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:12.575433 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3cc76-6fe6-4b44-b9fa-018ad13fc4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:12.576021 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3cc76-6fe6-4b44-b9fa-018ad13fc4ce admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:12.576296 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3cc76-6fe6-4b44-b9fa-018ad13fc4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:12.576561 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3cc76-6fe6-4b44-b9fa-018ad13fc4ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:12.577189 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b3cc76-6fe6-4b44-b9fa-018ad13fc4ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:12.578756 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b3cc76-6fe6-4b44-b9fa-018ad13fc4ce admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:12.589461 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1380/2759] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:43:12.593420 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe20fc-3066-48ae-ac95-c30592f52ca4 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:12.603292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe20fc-3066-48ae-ac95-c30592f52ca4 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:12.607847 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a6a27b-50aa-433f-a419-1649ecbbb271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:12.608435 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6a27b-50aa-433f-a419-1649ecbbb271 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:12.608717 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6a27b-50aa-433f-a419-1649ecbbb271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:12.608991 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6a27b-50aa-433f-a419-1649ecbbb271 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:12.609658 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a6a27b-50aa-433f-a419-1649ecbbb271 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:12.611278 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a6a27b-50aa-433f-a419-1649ecbbb271 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:12.620952 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1380/2760] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:43:12.624763 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3cc76-6fe6-4b44-b9fa-018ad13fc4ce admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:12.634729 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3cc76-6fe6-4b44-b9fa-018ad13fc4ce admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:12.639173 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed3f218-00f3-4e1c-9e65-34e62684996e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:12.639795 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3f218-00f3-4e1c-9e65-34e62684996e admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:12.640069 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3f218-00f3-4e1c-9e65-34e62684996e admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:12.640338 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3f218-00f3-4e1c-9e65-34e62684996e admin admin] PATH_INFO: `/v3/projects/2603ca8a11354de48d67781c9403be97/users/98d953e7698d410c8e46f72b65eb86b8/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:12.655337 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed3f218-00f3-4e1c-9e65-34e62684996e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2603ca8a11354de48d67781c9403be97, user_id=98d953e7698d410c8e46f72b65eb86b8, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:12.657952 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed3f218-00f3-4e1c-9e65-34e62684996e admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:12.670820 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1381/2761] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:43:12 2022] PUT /identity/v3/projects/2603ca8a11354de48d67781c9403be97/users/98d953e7698d410c8e46f72b65eb86b8/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:43:12.682874 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64915637-87ff-4c0e-b2d6-8627282d8775 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:12.683163 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64915637-87ff-4c0e-b2d6-8627282d8775 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:12.683437 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64915637-87ff-4c0e-b2d6-8627282d8775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:12.712314 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-64915637-87ff-4c0e-b2d6-8627282d8775 None None] MFA Rules not processed for user `98d953e7698d410c8e46f72b65eb86b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:43:12.713512 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-64915637-87ff-4c0e-b2d6-8627282d8775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:43:12.775406 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1381/2762] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:43:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:43:12.779865 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed3f218-00f3-4e1c-9e65-34e62684996e admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:12.841168 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed3f218-00f3-4e1c-9e65-34e62684996e admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:12.845819 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c5b752-e008-48a4-b460-e1a594b3c2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:12.846422 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c5b752-e008-48a4-b460-e1a594b3c2a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:12.846702 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c5b752-e008-48a4-b460-e1a594b3c2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:12.847004 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c5b752-e008-48a4-b460-e1a594b3c2a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:12.847632 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c5b752-e008-48a4-b460-e1a594b3c2a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:12.849609 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c5b752-e008-48a4-b460-e1a594b3c2a8 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:12.869860 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:43:12.878376 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1382/2763] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:43:12 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:43:12.883031 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64915637-87ff-4c0e-b2d6-8627282d8775 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:12.893694 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64915637-87ff-4c0e-b2d6-8627282d8775 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:12.948704 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e66cd-90ac-4fa2-8cb8-d1fac1c9475a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:12.949277 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e66cd-90ac-4fa2-8cb8-d1fac1c9475a admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:12.949554 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e66cd-90ac-4fa2-8cb8-d1fac1c9475a admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:12.949826 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e66cd-90ac-4fa2-8cb8-d1fac1c9475a admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:12.950446 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e66cd-90ac-4fa2-8cb8-d1fac1c9475a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:12.952388 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e66cd-90ac-4fa2-8cb8-d1fac1c9475a admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:12.973138 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1382/2764] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:43:12 2022] POST /identity/v3/users => generated 296 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:43:12.977127 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c5b752-e008-48a4-b460-e1a594b3c2a8 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:12.987097 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c5b752-e008-48a4-b460-e1a594b3c2a8 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:12.992069 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e16329-f167-4ea4-8480-fc64b5923cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:12.992654 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e16329-f167-4ea4-8480-fc64b5923cb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:12.992927 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e16329-f167-4ea4-8480-fc64b5923cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:12.993196 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e16329-f167-4ea4-8480-fc64b5923cb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:12.993824 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e16329-f167-4ea4-8480-fc64b5923cb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:12.995420 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e16329-f167-4ea4-8480-fc64b5923cb4 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:13.005668 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1383/2765] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:43:13.010043 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e66cd-90ac-4fa2-8cb8-d1fac1c9475a admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:13.019885 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e66cd-90ac-4fa2-8cb8-d1fac1c9475a admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:13.024526 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63efe1f-b28e-447e-b4d2-aec658fa84e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:13.025150 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63efe1f-b28e-447e-b4d2-aec658fa84e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:13.025436 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63efe1f-b28e-447e-b4d2-aec658fa84e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:13.025712 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63efe1f-b28e-447e-b4d2-aec658fa84e1 admin admin] PATH_INFO: `/v3/projects/81e242894c1244c0bafc34eb82cc5bbc/users/63293348521542ab9b82f3677f4de273/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:13.040454 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63efe1f-b28e-447e-b4d2-aec658fa84e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81e242894c1244c0bafc34eb82cc5bbc, user_id=63293348521542ab9b82f3677f4de273, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:13.043104 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63efe1f-b28e-447e-b4d2-aec658fa84e1 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:13.056360 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1383/2766] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:43:13 2022] PUT /identity/v3/projects/81e242894c1244c0bafc34eb82cc5bbc/users/63293348521542ab9b82f3677f4de273/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:43:13.060196 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e16329-f167-4ea4-8480-fc64b5923cb4 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:13.070499 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e16329-f167-4ea4-8480-fc64b5923cb4 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:13.125519 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1940224f-467e-4f67-be94-2399a2c2af60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:13.126094 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1940224f-467e-4f67-be94-2399a2c2af60 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:13.126372 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1940224f-467e-4f67-be94-2399a2c2af60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:13.126643 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1940224f-467e-4f67-be94-2399a2c2af60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:13.127331 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1940224f-467e-4f67-be94-2399a2c2af60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:13.128903 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1940224f-467e-4f67-be94-2399a2c2af60 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:13.138722 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1384/2767] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:43:13 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:43:13.142754 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63efe1f-b28e-447e-b4d2-aec658fa84e1 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:13.155178 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63efe1f-b28e-447e-b4d2-aec658fa84e1 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:13.159758 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59eeff-52a0-4cd0-8687-5c9191b3ac3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:13.160402 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59eeff-52a0-4cd0-8687-5c9191b3ac3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:13.160682 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59eeff-52a0-4cd0-8687-5c9191b3ac3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:13.160955 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59eeff-52a0-4cd0-8687-5c9191b3ac3a admin admin] PATH_INFO: `/v3/projects/81e242894c1244c0bafc34eb82cc5bbc/users/63293348521542ab9b82f3677f4de273/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:13.164382 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d59eeff-52a0-4cd0-8687-5c9191b3ac3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=81e242894c1244c0bafc34eb82cc5bbc, user_id=63293348521542ab9b82f3677f4de273, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:13.167870 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d59eeff-52a0-4cd0-8687-5c9191b3ac3a admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:13.174076 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:43:13.181215 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1384/2768] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:43:13 2022] PUT /identity/v3/projects/81e242894c1244c0bafc34eb82cc5bbc/users/63293348521542ab9b82f3677f4de273/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:43:13.242045 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fefe45-17a0-4cc3-b112-580db4901936 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:13.242325 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fefe45-17a0-4cc3-b112-580db4901936 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:13.242599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fefe45-17a0-4cc3-b112-580db4901936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:13.270215 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-58fefe45-17a0-4cc3-b112-580db4901936 None None] MFA Rules not processed for user `63293348521542ab9b82f3677f4de273`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:43:13.271430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-58fefe45-17a0-4cc3-b112-580db4901936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:43:13.337994 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1385/2769] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:43:13 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:43:13.348523 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59eeff-52a0-4cd0-8687-5c9191b3ac3a admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:13.350698 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-4d59eeff-52a0-4cd0-8687-5c9191b3ac3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:43:13.407497 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59eeff-52a0-4cd0-8687-5c9191b3ac3a admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:13.411111 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2486ee96-14b2-4353-a495-94490a778a07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:13.412449 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486ee96-14b2-4353-a495-94490a778a07 service cinder] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:13.412733 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486ee96-14b2-4353-a495-94490a778a07 service cinder] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:13.413009 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486ee96-14b2-4353-a495-94490a778a07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:13.421750 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2486ee96-14b2-4353-a495-94490a778a07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:13.423196 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2486ee96-14b2-4353-a495-94490a778a07 service cinder] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:13.432979 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1385/2770] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:43:13 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:43:13.796252 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fefe45-17a0-4cc3-b112-580db4901936 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:13.856415 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fefe45-17a0-4cc3-b112-580db4901936 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:13.860883 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9052873-8089-4728-ad78-18d697f0c10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:13.861512 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9052873-8089-4728-ad78-18d697f0c10e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:13.861787 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9052873-8089-4728-ad78-18d697f0c10e admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:13.862056 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9052873-8089-4728-ad78-18d697f0c10e admin admin] PATH_INFO: `/v3/users/98d953e7698d410c8e46f72b65eb86b8` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:13.863207 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9052873-8089-4728-ad78-18d697f0c10e admin admin] RBAC: Authorizing `identity:delete_user(user_id=98d953e7698d410c8e46f72b65eb86b8)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:13.865098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9052873-8089-4728-ad78-18d697f0c10e admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:13.875557 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:43:13.910585 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d9052873-8089-4728-ad78-18d697f0c10e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98d953e7698d410c8e46f72b65eb86b8'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:13.911209 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d9052873-8089-4728-ad78-18d697f0c10e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98d953e7698d410c8e46f72b65eb86b8'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:13.912085 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d9052873-8089-4728-ad78-18d697f0c10e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98d953e7698d410c8e46f72b65eb86b8'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:13.923965 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d9052873-8089-4728-ad78-18d697f0c10e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98d953e7698d410c8e46f72b65eb86b8'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:13.930828 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d9052873-8089-4728-ad78-18d697f0c10e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98d953e7698d410c8e46f72b65eb86b8'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:13.942916 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1386/2771] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:43:13 2022] DELETE /identity/v3/users/98d953e7698d410c8e46f72b65eb86b8 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:43:13.947959 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f5a5d-1220-423e-8d67-c2d7187d3c60 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:13.948428 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f5a5d-1220-423e-8d67-c2d7187d3c60 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:13.948550 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f5a5d-1220-423e-8d67-c2d7187d3c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:13.967370 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-255f5a5d-1220-423e-8d67-c2d7187d3c60 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:43:13.968545 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-255f5a5d-1220-423e-8d67-c2d7187d3c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:43:14.027891 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1386/2772] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:43:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:43:14.037800 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9052873-8089-4728-ad78-18d697f0c10e admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:14.039331 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-d9052873-8089-4728-ad78-18d697f0c10e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:43:14.095659 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9052873-8089-4728-ad78-18d697f0c10e admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:14.099355 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c961e20-2e59-44b1-9eb3-0e98b91b999f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:14.099957 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c961e20-2e59-44b1-9eb3-0e98b91b999f service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:14.100233 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c961e20-2e59-44b1-9eb3-0e98b91b999f service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:14.100504 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c961e20-2e59-44b1-9eb3-0e98b91b999f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:14.109365 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c961e20-2e59-44b1-9eb3-0e98b91b999f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:14.110789 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c961e20-2e59-44b1-9eb3-0e98b91b999f service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:14.121063 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1387/2773] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:43:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:43:14.378461 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255f5a5d-1220-423e-8d67-c2d7187d3c60 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:14.379998 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-255f5a5d-1220-423e-8d67-c2d7187d3c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:43:14.394858 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255f5a5d-1220-423e-8d67-c2d7187d3c60 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:14.400493 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f668b2c2-db5f-4de8-b210-e7d149c0fad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:14.401119 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668b2c2-db5f-4de8-b210-e7d149c0fad9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:14.401402 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668b2c2-db5f-4de8-b210-e7d149c0fad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:14.401742 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668b2c2-db5f-4de8-b210-e7d149c0fad9 admin admin] PATH_INFO: `/v3/projects/2603ca8a11354de48d67781c9403be97` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:14.403074 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f668b2c2-db5f-4de8-b210-e7d149c0fad9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2603ca8a11354de48d67781c9403be97)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:14.404959 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f668b2c2-db5f-4de8-b210-e7d149c0fad9 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:14.451053 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-f668b2c2-db5f-4de8-b210-e7d149c0fad9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2603ca8a11354de48d67781c9403be97', 'request_id': 'req-f668b2c2-db5f-4de8-b210-e7d149c0fad9'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:14.456954 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-f668b2c2-db5f-4de8-b210-e7d149c0fad9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2603ca8a11354de48d67781c9403be97', 'request_id': 'req-f668b2c2-db5f-4de8-b210-e7d149c0fad9'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:14.462427 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-f668b2c2-db5f-4de8-b210-e7d149c0fad9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2603ca8a11354de48d67781c9403be97', 'request_id': 'req-f668b2c2-db5f-4de8-b210-e7d149c0fad9'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:14.473953 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-f668b2c2-db5f-4de8-b210-e7d149c0fad9 admin admin] The token cache is being invalidate because project 2603ca8a11354de48d67781c9403be97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:43:14.474236 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-f668b2c2-db5f-4de8-b210-e7d149c0fad9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:14.475002 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-f668b2c2-db5f-4de8-b210-e7d149c0fad9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:14.476431 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1387/2774] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:43:14 2022] DELETE /identity/v3/projects/2603ca8a11354de48d67781c9403be97 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:43:14.480462 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c961e20-2e59-44b1-9eb3-0e98b91b999f service neutron] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:14.490011 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c961e20-2e59-44b1-9eb3-0e98b91b999f service neutron] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:14.545307 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5014fe-7665-4703-9733-331afc3efd7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:14.545931 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5014fe-7665-4703-9733-331afc3efd7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:14.546207 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5014fe-7665-4703-9733-331afc3efd7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:14.546477 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5014fe-7665-4703-9733-331afc3efd7b admin admin] PATH_INFO: `/v3/users/63293348521542ab9b82f3677f4de273` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:14.547617 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5014fe-7665-4703-9733-331afc3efd7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=63293348521542ab9b82f3677f4de273)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:14.549550 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5014fe-7665-4703-9733-331afc3efd7b admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:14.584214 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:43:14.597601 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-3e5014fe-7665-4703-9733-331afc3efd7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63293348521542ab9b82f3677f4de273'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:14.598242 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-3e5014fe-7665-4703-9733-331afc3efd7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63293348521542ab9b82f3677f4de273'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:14.598868 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-3e5014fe-7665-4703-9733-331afc3efd7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63293348521542ab9b82f3677f4de273'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:14.610978 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-3e5014fe-7665-4703-9733-331afc3efd7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63293348521542ab9b82f3677f4de273'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:14.618635 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-3e5014fe-7665-4703-9733-331afc3efd7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63293348521542ab9b82f3677f4de273'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:14.630747 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1388/2775] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:43:14 2022] DELETE /identity/v3/users/63293348521542ab9b82f3677f4de273 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:43:15.166467 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f668b2c2-db5f-4de8-b210-e7d149c0fad9 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:15.168216 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-f668b2c2-db5f-4de8-b210-e7d149c0fad9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:43:15.184750 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f668b2c2-db5f-4de8-b210-e7d149c0fad9 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:15.205941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:43:15.245220 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff6d34-955f-42b8-b6a0-4e6a3726bc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:15.245857 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff6d34-955f-42b8-b6a0-4e6a3726bc6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:15.246135 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff6d34-955f-42b8-b6a0-4e6a3726bc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:15.246405 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff6d34-955f-42b8-b6a0-4e6a3726bc6f admin admin] PATH_INFO: `/v3/projects/81e242894c1244c0bafc34eb82cc5bbc` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:15.247532 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff6d34-955f-42b8-b6a0-4e6a3726bc6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=81e242894c1244c0bafc34eb82cc5bbc)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:15.249421 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff6d34-955f-42b8-b6a0-4e6a3726bc6f admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:15.299455 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-cbff6d34-955f-42b8-b6a0-4e6a3726bc6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81e242894c1244c0bafc34eb82cc5bbc', 'request_id': 'req-cbff6d34-955f-42b8-b6a0-4e6a3726bc6f'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:15.305697 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-cbff6d34-955f-42b8-b6a0-4e6a3726bc6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81e242894c1244c0bafc34eb82cc5bbc', 'request_id': 'req-cbff6d34-955f-42b8-b6a0-4e6a3726bc6f'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:15.311486 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-cbff6d34-955f-42b8-b6a0-4e6a3726bc6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81e242894c1244c0bafc34eb82cc5bbc', 'request_id': 'req-cbff6d34-955f-42b8-b6a0-4e6a3726bc6f'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:15.323613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-cbff6d34-955f-42b8-b6a0-4e6a3726bc6f admin admin] The token cache is being invalidate because project 81e242894c1244c0bafc34eb82cc5bbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:43:15.323897 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-cbff6d34-955f-42b8-b6a0-4e6a3726bc6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:15.324510 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-cbff6d34-955f-42b8-b6a0-4e6a3726bc6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:15.325993 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1388/2776] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:43:15 2022] DELETE /identity/v3/projects/81e242894c1244c0bafc34eb82cc5bbc => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:43:15.330803 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdcca8d-7300-4dae-9ce5-395aa846efb8 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:15.331290 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdcca8d-7300-4dae-9ce5-395aa846efb8 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:15.331415 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdcca8d-7300-4dae-9ce5-395aa846efb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:15.351361 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-ecdcca8d-7300-4dae-9ce5-395aa846efb8 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:43:15.352550 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-ecdcca8d-7300-4dae-9ce5-395aa846efb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:43:15.409976 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1389/2777] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:43:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:43:15.418058 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594dd1f-3139-4ea3-ae17-09df480c71f7 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:15.418411 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594dd1f-3139-4ea3-ae17-09df480c71f7 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:15.418684 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594dd1f-3139-4ea3-ae17-09df480c71f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:15.437982 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-c594dd1f-3139-4ea3-ae17-09df480c71f7 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:43:15.439204 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-c594dd1f-3139-4ea3-ae17-09df480c71f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:43:15.447438 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1389/2778] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:43:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:43:15.451457 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdcca8d-7300-4dae-9ce5-395aa846efb8 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:15.461499 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdcca8d-7300-4dae-9ce5-395aa846efb8 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:15.466069 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a4f586-22b7-4911-9745-a5a835766718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:15.466661 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a4f586-22b7-4911-9745-a5a835766718 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:15.466975 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a4f586-22b7-4911-9745-a5a835766718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:15.467246 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a4f586-22b7-4911-9745-a5a835766718 admin admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:15.467882 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a4f586-22b7-4911-9745-a5a835766718 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:43:15.468181 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a4f586-22b7-4911-9745-a5a835766718 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:15.469762 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a4f586-22b7-4911-9745-a5a835766718 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:15.480951 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1390/2779] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:43:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:43:15.484975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c594dd1f-3139-4ea3-ae17-09df480c71f7 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:15.494027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c594dd1f-3139-4ea3-ae17-09df480c71f7 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:15.498559 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75657de2-92ef-4d68-8477-a5a7b07bd5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:15.499177 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75657de2-92ef-4d68-8477-a5a7b07bd5b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:15.499454 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75657de2-92ef-4d68-8477-a5a7b07bd5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:15.499719 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75657de2-92ef-4d68-8477-a5a7b07bd5b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:15.500345 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75657de2-92ef-4d68-8477-a5a7b07bd5b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:15.502214 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75657de2-92ef-4d68-8477-a5a7b07bd5b5 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:15.529109 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1390/2780] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:43:15 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:43:15.533218 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a4f586-22b7-4911-9745-a5a835766718 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:15.543169 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a4f586-22b7-4911-9745-a5a835766718 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:15.602626 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc8d7e1-b318-4078-b431-e587d38b9e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:15.603333 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8d7e1-b318-4078-b431-e587d38b9e26 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:15.603639 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8d7e1-b318-4078-b431-e587d38b9e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:15.603906 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8d7e1-b318-4078-b431-e587d38b9e26 admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:15.604540 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc8d7e1-b318-4078-b431-e587d38b9e26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:15.606463 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc8d7e1-b318-4078-b431-e587d38b9e26 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:15.627176 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1391/2781] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:43:15 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:43:15.631856 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75657de2-92ef-4d68-8477-a5a7b07bd5b5 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:15.641839 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75657de2-92ef-4d68-8477-a5a7b07bd5b5 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:15.646305 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd5d16-a454-43bd-9828-ef55a94bd2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:15.647545 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd5d16-a454-43bd-9828-ef55a94bd2e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:15.647816 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd5d16-a454-43bd-9828-ef55a94bd2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:15.648092 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd5d16-a454-43bd-9828-ef55a94bd2e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:15.648721 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fd5d16-a454-43bd-9828-ef55a94bd2e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:15.650283 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fd5d16-a454-43bd-9828-ef55a94bd2e8 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:15.659899 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1391/2782] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:43:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:43:15.664015 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc8d7e1-b318-4078-b431-e587d38b9e26 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:15.673599 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc8d7e1-b318-4078-b431-e587d38b9e26 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:15.678137 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3228dc21-ad3e-4073-a0ca-8d63f5970d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:15.678721 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3228dc21-ad3e-4073-a0ca-8d63f5970d7a admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:15.679022 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3228dc21-ad3e-4073-a0ca-8d63f5970d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:15.679289 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3228dc21-ad3e-4073-a0ca-8d63f5970d7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:15.679917 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3228dc21-ad3e-4073-a0ca-8d63f5970d7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:15.681557 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3228dc21-ad3e-4073-a0ca-8d63f5970d7a admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:15.691847 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1392/2783] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:43:15 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 14:43:15.696238 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd5d16-a454-43bd-9828-ef55a94bd2e8 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:15.706136 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd5d16-a454-43bd-9828-ef55a94bd2e8 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:15.711224 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a7c4e6-4ede-4899-b847-e22188e6667d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:15.711859 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a7c4e6-4ede-4899-b847-e22188e6667d admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:15.712130 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a7c4e6-4ede-4899-b847-e22188e6667d admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:15.712398 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a7c4e6-4ede-4899-b847-e22188e6667d admin admin] PATH_INFO: `/v3/projects/7075233a4e0c4ac5a9f1767a0d50bb2d/users/71c68bf720824232bee72317c997c4cb/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:15.726528 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a7c4e6-4ede-4899-b847-e22188e6667d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7075233a4e0c4ac5a9f1767a0d50bb2d, user_id=71c68bf720824232bee72317c997c4cb, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:15.729161 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a7c4e6-4ede-4899-b847-e22188e6667d admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:15.741732 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1392/2784] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:43:15 2022] PUT /identity/v3/projects/7075233a4e0c4ac5a9f1767a0d50bb2d/users/71c68bf720824232bee72317c997c4cb/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:43:15.752992 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c3a54-891c-4f47-b06e-0093699d590b None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:15.753274 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c3a54-891c-4f47-b06e-0093699d590b None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:15.753544 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c3a54-891c-4f47-b06e-0093699d590b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:15.783513 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-368c3a54-891c-4f47-b06e-0093699d590b None None] MFA Rules not processed for user `71c68bf720824232bee72317c997c4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:43:15.784708 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-368c3a54-891c-4f47-b06e-0093699d590b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:43:15.788795 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:43:15.851717 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1393/2785] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:43:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:43:15.856376 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a7c4e6-4ede-4899-b847-e22188e6667d admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:15.906602 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:43:15.919364 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a7c4e6-4ede-4899-b847-e22188e6667d admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:15.924403 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d4d17-4fe7-44b1-a190-8fea90e34684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:15.925006 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d4d17-4fe7-44b1-a190-8fea90e34684 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:15.925282 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d4d17-4fe7-44b1-a190-8fea90e34684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:15.925550 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d4d17-4fe7-44b1-a190-8fea90e34684 admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:15.926176 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5d4d17-4fe7-44b1-a190-8fea90e34684 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:15.928591 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5d4d17-4fe7-44b1-a190-8fea90e34684 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:15.957039 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1393/2786] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:43:15 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:43:15.961115 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c3a54-891c-4f47-b06e-0093699d590b None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:15.971695 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c3a54-891c-4f47-b06e-0093699d590b None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:16.027292 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e4ad9-11ff-4d0f-97e0-0b6173fa7d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:16.027887 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e4ad9-11ff-4d0f-97e0-0b6173fa7d51 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:16.028167 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e4ad9-11ff-4d0f-97e0-0b6173fa7d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:16.028437 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e4ad9-11ff-4d0f-97e0-0b6173fa7d51 admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:16.029064 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e4ad9-11ff-4d0f-97e0-0b6173fa7d51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:16.031026 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e4ad9-11ff-4d0f-97e0-0b6173fa7d51 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:16.051855 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1394/2787] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:43:15 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:43:16.056346 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d4d17-4fe7-44b1-a190-8fea90e34684 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:16.067027 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d4d17-4fe7-44b1-a190-8fea90e34684 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:16.072199 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468df5b9-a8dd-45bb-8021-9e6bb5d9103c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:16.072801 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468df5b9-a8dd-45bb-8021-9e6bb5d9103c admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:16.073080 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468df5b9-a8dd-45bb-8021-9e6bb5d9103c admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:16.073353 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468df5b9-a8dd-45bb-8021-9e6bb5d9103c admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:16.073992 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468df5b9-a8dd-45bb-8021-9e6bb5d9103c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:16.075608 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468df5b9-a8dd-45bb-8021-9e6bb5d9103c admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:16.086565 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1394/2788] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:43:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:43:16.090454 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e4ad9-11ff-4d0f-97e0-0b6173fa7d51 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:16.101012 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e4ad9-11ff-4d0f-97e0-0b6173fa7d51 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:16.105671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f136be0-01af-40d9-b5ca-e25be1f74f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:16.106265 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f136be0-01af-40d9-b5ca-e25be1f74f57 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:16.106539 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f136be0-01af-40d9-b5ca-e25be1f74f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:16.106805 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f136be0-01af-40d9-b5ca-e25be1f74f57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:16.107459 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f136be0-01af-40d9-b5ca-e25be1f74f57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:16.109014 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f136be0-01af-40d9-b5ca-e25be1f74f57 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:16.122641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1395/2789] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:43:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:43:16.123767 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468df5b9-a8dd-45bb-8021-9e6bb5d9103c admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:16.133866 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468df5b9-a8dd-45bb-8021-9e6bb5d9103c admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:16.138518 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4667de47-fadf-42d9-ac2c-f0ed7afd7c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:16.139192 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667de47-fadf-42d9-ac2c-f0ed7afd7c80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:16.139462 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667de47-fadf-42d9-ac2c-f0ed7afd7c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:16.139728 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667de47-fadf-42d9-ac2c-f0ed7afd7c80 admin admin] PATH_INFO: `/v3/projects/4a5f53b957a24005ac92d56c6aea56cf/users/a38cbae0303f49ac8055883c3caf414f/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:16.154660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4667de47-fadf-42d9-ac2c-f0ed7afd7c80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a5f53b957a24005ac92d56c6aea56cf, user_id=a38cbae0303f49ac8055883c3caf414f, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:16.157279 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4667de47-fadf-42d9-ac2c-f0ed7afd7c80 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:16.172093 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1395/2790] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:43:16 2022] PUT /identity/v3/projects/4a5f53b957a24005ac92d56c6aea56cf/users/a38cbae0303f49ac8055883c3caf414f/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:43:16.183610 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87cbf5f-4f38-4306-b3bd-7c812f081653 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:16.184005 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87cbf5f-4f38-4306-b3bd-7c812f081653 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:16.184341 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87cbf5f-4f38-4306-b3bd-7c812f081653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:16.212837 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-b87cbf5f-4f38-4306-b3bd-7c812f081653 None None] MFA Rules not processed for user `a38cbae0303f49ac8055883c3caf414f`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:43:16.214053 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-b87cbf5f-4f38-4306-b3bd-7c812f081653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:43:16.281958 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1396/2791] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:43:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:43:16.286265 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4667de47-fadf-42d9-ac2c-f0ed7afd7c80 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:16.349583 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4667de47-fadf-42d9-ac2c-f0ed7afd7c80 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:16.355008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923746d8-aafe-4577-9890-2105ee590dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:16.355645 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923746d8-aafe-4577-9890-2105ee590dd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:16.355920 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923746d8-aafe-4577-9890-2105ee590dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:16.356325 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923746d8-aafe-4577-9890-2105ee590dd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:16.357051 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923746d8-aafe-4577-9890-2105ee590dd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:16.360306 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923746d8-aafe-4577-9890-2105ee590dd7 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:16.389689 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1396/2792] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:43:16 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:43:16.394513 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87cbf5f-4f38-4306-b3bd-7c812f081653 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:16.405737 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87cbf5f-4f38-4306-b3bd-7c812f081653 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:16.418113 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:43:16.467521 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced7a8b8-80db-4da2-9494-1d6a6bf5f2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:16.468171 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced7a8b8-80db-4da2-9494-1d6a6bf5f2b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:16.468453 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced7a8b8-80db-4da2-9494-1d6a6bf5f2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:16.468727 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced7a8b8-80db-4da2-9494-1d6a6bf5f2b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:16.469358 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced7a8b8-80db-4da2-9494-1d6a6bf5f2b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:16.471469 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced7a8b8-80db-4da2-9494-1d6a6bf5f2b7 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:16.496934 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1397/2793] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:43:16 2022] POST /identity/v3/users => generated 298 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:43:16.498121 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923746d8-aafe-4577-9890-2105ee590dd7 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:16.508501 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923746d8-aafe-4577-9890-2105ee590dd7 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:16.513564 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f95d3-f0e6-4000-8a62-ac96012cdd35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:16.514165 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f95d3-f0e6-4000-8a62-ac96012cdd35 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:16.514440 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f95d3-f0e6-4000-8a62-ac96012cdd35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:16.514712 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f95d3-f0e6-4000-8a62-ac96012cdd35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:16.515505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f95d3-f0e6-4000-8a62-ac96012cdd35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:16.517095 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f95d3-f0e6-4000-8a62-ac96012cdd35 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:16.531022 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1397/2794] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:43:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:43:16.534902 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced7a8b8-80db-4da2-9494-1d6a6bf5f2b7 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:16.545773 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced7a8b8-80db-4da2-9494-1d6a6bf5f2b7 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:16.550388 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e535749e-9830-4d69-be1c-c23914985394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:16.551076 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535749e-9830-4d69-be1c-c23914985394 admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:16.551360 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535749e-9830-4d69-be1c-c23914985394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:16.551634 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535749e-9830-4d69-be1c-c23914985394 admin admin] PATH_INFO: `/v3/projects/fdb9dce8fdf141f3a33ae1a8becc6784/users/bdd74b743677472eb117b11b33010d48/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:16.566015 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e535749e-9830-4d69-be1c-c23914985394 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdb9dce8fdf141f3a33ae1a8becc6784, user_id=bdd74b743677472eb117b11b33010d48, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:16.568699 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e535749e-9830-4d69-be1c-c23914985394 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:16.581872 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1398/2795] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:43:16 2022] PUT /identity/v3/projects/fdb9dce8fdf141f3a33ae1a8becc6784/users/bdd74b743677472eb117b11b33010d48/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:43:16.586355 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f95d3-f0e6-4000-8a62-ac96012cdd35 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:16.596261 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f95d3-f0e6-4000-8a62-ac96012cdd35 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:16.668246 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae73d76-78bc-403d-b524-b9443c961a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:16.668849 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae73d76-78bc-403d-b524-b9443c961a9e admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:16.669158 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae73d76-78bc-403d-b524-b9443c961a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:16.669427 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae73d76-78bc-403d-b524-b9443c961a9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:16.670109 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae73d76-78bc-403d-b524-b9443c961a9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:16.671744 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae73d76-78bc-403d-b524-b9443c961a9e admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:16.681342 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1398/2796] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:43:16 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:43:16.685537 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e535749e-9830-4d69-be1c-c23914985394 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:16.695671 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e535749e-9830-4d69-be1c-c23914985394 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:16.700260 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2668cbef-b4bf-4b9c-9b98-c5504ae79632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:16.700900 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2668cbef-b4bf-4b9c-9b98-c5504ae79632 admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:16.701200 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2668cbef-b4bf-4b9c-9b98-c5504ae79632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:16.701453 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2668cbef-b4bf-4b9c-9b98-c5504ae79632 admin admin] PATH_INFO: `/v3/projects/fdb9dce8fdf141f3a33ae1a8becc6784/users/bdd74b743677472eb117b11b33010d48/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:16.703487 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2668cbef-b4bf-4b9c-9b98-c5504ae79632 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdb9dce8fdf141f3a33ae1a8becc6784, user_id=bdd74b743677472eb117b11b33010d48, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:16.706122 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2668cbef-b4bf-4b9c-9b98-c5504ae79632 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:16.716734 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1399/2797] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:43:16 2022] PUT /identity/v3/projects/fdb9dce8fdf141f3a33ae1a8becc6784/users/bdd74b743677472eb117b11b33010d48/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:43:16.727442 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d30f41-00e1-4b86-9c4e-d0af6c43673c None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:16.727793 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d30f41-00e1-4b86-9c4e-d0af6c43673c None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:16.728064 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d30f41-00e1-4b86-9c4e-d0af6c43673c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:16.747751 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:43:16.755263 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-f5d30f41-00e1-4b86-9c4e-d0af6c43673c None None] MFA Rules not processed for user `bdd74b743677472eb117b11b33010d48`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:43:16.757020 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-f5d30f41-00e1-4b86-9c4e-d0af6c43673c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:43:16.823359 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1399/2798] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:43:16 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:43:16.835089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2668cbef-b4bf-4b9c-9b98-c5504ae79632 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:16.836834 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-2668cbef-b4bf-4b9c-9b98-c5504ae79632 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:43:16.895315 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2668cbef-b4bf-4b9c-9b98-c5504ae79632 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:16.899621 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade8a0c6-057a-4376-8ed5-a49ecec306fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:16.900220 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8a0c6-057a-4376-8ed5-a49ecec306fe service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:16.900500 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8a0c6-057a-4376-8ed5-a49ecec306fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:16.900776 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8a0c6-057a-4376-8ed5-a49ecec306fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:16.955856 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade8a0c6-057a-4376-8ed5-a49ecec306fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:16.957298 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade8a0c6-057a-4376-8ed5-a49ecec306fe service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:16.966687 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1400/2799] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:43:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:43:18.489371 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d30f41-00e1-4b86-9c4e-d0af6c43673c None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:18.490876 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-f5d30f41-00e1-4b86-9c4e-d0af6c43673c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:43:18.502983 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d30f41-00e1-4b86-9c4e-d0af6c43673c None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:18.507422 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6004d469-2592-4688-a65b-a913daa97703 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:18.508081 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004d469-2592-4688-a65b-a913daa97703 service cinder] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:18.508359 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004d469-2592-4688-a65b-a913daa97703 service cinder] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:18.508629 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004d469-2592-4688-a65b-a913daa97703 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:18.563401 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6004d469-2592-4688-a65b-a913daa97703 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:18.564871 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6004d469-2592-4688-a65b-a913daa97703 service cinder] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:18.574198 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1400/2800] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:43:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:43:19.125917 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade8a0c6-057a-4376-8ed5-a49ecec306fe service cinder] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:19.134626 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade8a0c6-057a-4376-8ed5-a49ecec306fe service cinder] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:19.138353 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f61db-a946-4075-819e-439b115e29ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:19.138993 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f61db-a946-4075-819e-439b115e29ce service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:19.139294 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f61db-a946-4075-819e-439b115e29ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:19.139567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f61db-a946-4075-819e-439b115e29ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:19.151626 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f61db-a946-4075-819e-439b115e29ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:19.153156 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f61db-a946-4075-819e-439b115e29ce service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:19.163559 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1401/2801] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:43:19 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:43:22.923762 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32295b46-6a02-42dd-892e-903985438691 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:22.924255 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32295b46-6a02-42dd-892e-903985438691 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:22.924383 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32295b46-6a02-42dd-892e-903985438691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:22.942890 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-32295b46-6a02-42dd-892e-903985438691 None None] MFA Rules not processed for user `f5d7b40e78a049cd83d079291c3601ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:43:22.944588 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-32295b46-6a02-42dd-892e-903985438691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:43:22.997591 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1401/2802] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:43:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:43:23.086216 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f61db-a946-4075-819e-439b115e29ce service cinder] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:23.094985 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:43:23.149814 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f61db-a946-4075-819e-439b115e29ce service cinder] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:23.154350 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdbacdd-091d-4e23-96b7-e7c240f97cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:23.155021 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbacdd-091d-4e23-96b7-e7c240f97cd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:23.155303 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbacdd-091d-4e23-96b7-e7c240f97cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:23.155583 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbacdd-091d-4e23-96b7-e7c240f97cd5 admin admin] PATH_INFO: `/v3/users/71c68bf720824232bee72317c997c4cb` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:23.156693 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdbacdd-091d-4e23-96b7-e7c240f97cd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71c68bf720824232bee72317c997c4cb)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:23.158591 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdbacdd-091d-4e23-96b7-e7c240f97cd5 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:23.207065 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-cfdbacdd-091d-4e23-96b7-e7c240f97cd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71c68bf720824232bee72317c997c4cb'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:23.207776 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-cfdbacdd-091d-4e23-96b7-e7c240f97cd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71c68bf720824232bee72317c997c4cb'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:23.208375 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-cfdbacdd-091d-4e23-96b7-e7c240f97cd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71c68bf720824232bee72317c997c4cb'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:23.220763 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-cfdbacdd-091d-4e23-96b7-e7c240f97cd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71c68bf720824232bee72317c997c4cb'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:23.228705 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-cfdbacdd-091d-4e23-96b7-e7c240f97cd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71c68bf720824232bee72317c997c4cb'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:23.242710 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1402/2803] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:43:23 2022] DELETE /identity/v3/users/71c68bf720824232bee72317c997c4cb => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:43:23.247183 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39590a-e5ca-48c0-8474-9f4705d7c275 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:23.247534 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39590a-e5ca-48c0-8474-9f4705d7c275 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:23.247811 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39590a-e5ca-48c0-8474-9f4705d7c275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:23.266574 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-1f39590a-e5ca-48c0-8474-9f4705d7c275 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:43:23.267825 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-1f39590a-e5ca-48c0-8474-9f4705d7c275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:43:23.303516 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=100 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:43:23.329114 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1402/2804] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:43:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:43:23.339475 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdbacdd-091d-4e23-96b7-e7c240f97cd5 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:23.340911 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-cfdbacdd-091d-4e23-96b7-e7c240f97cd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:43:23.399902 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdbacdd-091d-4e23-96b7-e7c240f97cd5 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:23.403841 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77eecbb-c33e-4e5c-8c39-dc8d0a7d0733 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:23.404446 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77eecbb-c33e-4e5c-8c39-dc8d0a7d0733 service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:23.404720 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77eecbb-c33e-4e5c-8c39-dc8d0a7d0733 service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:23.404996 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77eecbb-c33e-4e5c-8c39-dc8d0a7d0733 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:23.415451 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77eecbb-c33e-4e5c-8c39-dc8d0a7d0733 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:23.416887 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77eecbb-c33e-4e5c-8c39-dc8d0a7d0733 service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:23.426566 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1403/2805] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:43:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:43:23.734509 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39590a-e5ca-48c0-8474-9f4705d7c275 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:23.736124 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-1f39590a-e5ca-48c0-8474-9f4705d7c275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:43:23.749079 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39590a-e5ca-48c0-8474-9f4705d7c275 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:23.760362 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f50b39-8066-499f-b2e3-a801a6c44f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:23.761012 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f50b39-8066-499f-b2e3-a801a6c44f4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:23.761700 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f50b39-8066-499f-b2e3-a801a6c44f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:23.761700 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f50b39-8066-499f-b2e3-a801a6c44f4f admin admin] PATH_INFO: `/v3/projects/7075233a4e0c4ac5a9f1767a0d50bb2d` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:23.762690 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f50b39-8066-499f-b2e3-a801a6c44f4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7075233a4e0c4ac5a9f1767a0d50bb2d)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:23.764671 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f50b39-8066-499f-b2e3-a801a6c44f4f admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:23.813404 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-f6f50b39-8066-499f-b2e3-a801a6c44f4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7075233a4e0c4ac5a9f1767a0d50bb2d', 'request_id': 'req-f6f50b39-8066-499f-b2e3-a801a6c44f4f'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:23.820798 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-f6f50b39-8066-499f-b2e3-a801a6c44f4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7075233a4e0c4ac5a9f1767a0d50bb2d', 'request_id': 'req-f6f50b39-8066-499f-b2e3-a801a6c44f4f'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:23.827488 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-f6f50b39-8066-499f-b2e3-a801a6c44f4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7075233a4e0c4ac5a9f1767a0d50bb2d', 'request_id': 'req-f6f50b39-8066-499f-b2e3-a801a6c44f4f'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:23.841031 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-f6f50b39-8066-499f-b2e3-a801a6c44f4f admin admin] The token cache is being invalidate because project 7075233a4e0c4ac5a9f1767a0d50bb2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:43:23.841324 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-f6f50b39-8066-499f-b2e3-a801a6c44f4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:23.841950 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-f6f50b39-8066-499f-b2e3-a801a6c44f4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:23.843462 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1403/2806] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:43:23 2022] DELETE /identity/v3/projects/7075233a4e0c4ac5a9f1767a0d50bb2d => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:43:23.847510 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77eecbb-c33e-4e5c-8c39-dc8d0a7d0733 service neutron] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:23.857571 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77eecbb-c33e-4e5c-8c39-dc8d0a7d0733 service neutron] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:23.886704 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:43:23.924101 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc20bff-61a9-4d5c-8284-c38721f72f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:23.924833 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc20bff-61a9-4d5c-8284-c38721f72f58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:23.925162 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc20bff-61a9-4d5c-8284-c38721f72f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:23.925502 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc20bff-61a9-4d5c-8284-c38721f72f58 admin admin] PATH_INFO: `/v3/users/a38cbae0303f49ac8055883c3caf414f` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:23.926699 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc20bff-61a9-4d5c-8284-c38721f72f58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a38cbae0303f49ac8055883c3caf414f)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:23.928970 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc20bff-61a9-4d5c-8284-c38721f72f58 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:23.979475 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9cc20bff-61a9-4d5c-8284-c38721f72f58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a38cbae0303f49ac8055883c3caf414f'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:23.980127 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9cc20bff-61a9-4d5c-8284-c38721f72f58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a38cbae0303f49ac8055883c3caf414f'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:23.980734 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9cc20bff-61a9-4d5c-8284-c38721f72f58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a38cbae0303f49ac8055883c3caf414f'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:23.997934 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9cc20bff-61a9-4d5c-8284-c38721f72f58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a38cbae0303f49ac8055883c3caf414f'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:24.007445 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9cc20bff-61a9-4d5c-8284-c38721f72f58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a38cbae0303f49ac8055883c3caf414f'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:24.020463 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1404/2807] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:43:23 2022] DELETE /identity/v3/users/a38cbae0303f49ac8055883c3caf414f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:43:24.295232 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f50b39-8066-499f-b2e3-a801a6c44f4f admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:24.297373 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-f6f50b39-8066-499f-b2e3-a801a6c44f4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:43:24.311682 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f50b39-8066-499f-b2e3-a801a6c44f4f admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:24.372745 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e26763-e78f-43c3-8d7b-c8e0df76870d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:24.373316 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e26763-e78f-43c3-8d7b-c8e0df76870d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:24.373593 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e26763-e78f-43c3-8d7b-c8e0df76870d admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:24.373861 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e26763-e78f-43c3-8d7b-c8e0df76870d admin admin] PATH_INFO: `/v3/projects/4a5f53b957a24005ac92d56c6aea56cf` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:24.375011 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e26763-e78f-43c3-8d7b-c8e0df76870d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a5f53b957a24005ac92d56c6aea56cf)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:24.376953 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e26763-e78f-43c3-8d7b-c8e0df76870d admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:24.426335 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-61e26763-e78f-43c3-8d7b-c8e0df76870d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a5f53b957a24005ac92d56c6aea56cf', 'request_id': 'req-61e26763-e78f-43c3-8d7b-c8e0df76870d'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:24.434058 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-61e26763-e78f-43c3-8d7b-c8e0df76870d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a5f53b957a24005ac92d56c6aea56cf', 'request_id': 'req-61e26763-e78f-43c3-8d7b-c8e0df76870d'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:24.439641 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-61e26763-e78f-43c3-8d7b-c8e0df76870d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a5f53b957a24005ac92d56c6aea56cf', 'request_id': 'req-61e26763-e78f-43c3-8d7b-c8e0df76870d'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:24.442115 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:43:24.453657 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-61e26763-e78f-43c3-8d7b-c8e0df76870d admin admin] The token cache is being invalidate because project 4a5f53b957a24005ac92d56c6aea56cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:43:24.453958 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-61e26763-e78f-43c3-8d7b-c8e0df76870d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:24.454807 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-61e26763-e78f-43c3-8d7b-c8e0df76870d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:24.456304 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1404/2808] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:43:24 2022] DELETE /identity/v3/projects/4a5f53b957a24005ac92d56c6aea56cf => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:43:24.460252 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc20bff-61a9-4d5c-8284-c38721f72f58 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:24.461633 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-9cc20bff-61a9-4d5c-8284-c38721f72f58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:43:24.474116 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc20bff-61a9-4d5c-8284-c38721f72f58 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:24.529259 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e7a906-b4f7-4200-a1f2-1a9b0c8fe333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:24.529883 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e7a906-b4f7-4200-a1f2-1a9b0c8fe333 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:24.530163 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e7a906-b4f7-4200-a1f2-1a9b0c8fe333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:24.530431 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e7a906-b4f7-4200-a1f2-1a9b0c8fe333 admin admin] PATH_INFO: `/v3/users/bdd74b743677472eb117b11b33010d48` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:24.531567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e7a906-b4f7-4200-a1f2-1a9b0c8fe333 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdd74b743677472eb117b11b33010d48)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:24.533461 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e7a906-b4f7-4200-a1f2-1a9b0c8fe333 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:24.579637 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-76e7a906-b4f7-4200-a1f2-1a9b0c8fe333 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdd74b743677472eb117b11b33010d48'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:24.580289 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-76e7a906-b4f7-4200-a1f2-1a9b0c8fe333 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdd74b743677472eb117b11b33010d48'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:24.580896 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-76e7a906-b4f7-4200-a1f2-1a9b0c8fe333 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdd74b743677472eb117b11b33010d48'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:24.582806 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:43:24.595041 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-76e7a906-b4f7-4200-a1f2-1a9b0c8fe333 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdd74b743677472eb117b11b33010d48'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:24.602339 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-76e7a906-b4f7-4200-a1f2-1a9b0c8fe333 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdd74b743677472eb117b11b33010d48'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:24.614731 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1405/2809] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:43:24 2022] DELETE /identity/v3/users/bdd74b743677472eb117b11b33010d48 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:43:24.928934 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e26763-e78f-43c3-8d7b-c8e0df76870d admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:24.930343 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-61e26763-e78f-43c3-8d7b-c8e0df76870d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:43:24.943639 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e26763-e78f-43c3-8d7b-c8e0df76870d admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:25.017088 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb23ee-5ac7-4ba7-afc4-83f314ffe408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:25.017789 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb23ee-5ac7-4ba7-afc4-83f314ffe408 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:25.018068 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb23ee-5ac7-4ba7-afc4-83f314ffe408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:25.018337 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb23ee-5ac7-4ba7-afc4-83f314ffe408 admin admin] PATH_INFO: `/v3/projects/fdb9dce8fdf141f3a33ae1a8becc6784` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:25.019534 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bb23ee-5ac7-4ba7-afc4-83f314ffe408 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdb9dce8fdf141f3a33ae1a8becc6784)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:25.022576 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bb23ee-5ac7-4ba7-afc4-83f314ffe408 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:25.069836 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-43bb23ee-5ac7-4ba7-afc4-83f314ffe408 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdb9dce8fdf141f3a33ae1a8becc6784', 'request_id': 'req-43bb23ee-5ac7-4ba7-afc4-83f314ffe408'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:25.075875 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-43bb23ee-5ac7-4ba7-afc4-83f314ffe408 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdb9dce8fdf141f3a33ae1a8becc6784', 'request_id': 'req-43bb23ee-5ac7-4ba7-afc4-83f314ffe408'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:25.081448 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-43bb23ee-5ac7-4ba7-afc4-83f314ffe408 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdb9dce8fdf141f3a33ae1a8becc6784', 'request_id': 'req-43bb23ee-5ac7-4ba7-afc4-83f314ffe408'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:25.093689 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-43bb23ee-5ac7-4ba7-afc4-83f314ffe408 admin admin] The token cache is being invalidate because project fdb9dce8fdf141f3a33ae1a8becc6784 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:43:25.093955 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-43bb23ee-5ac7-4ba7-afc4-83f314ffe408 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:25.094569 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-43bb23ee-5ac7-4ba7-afc4-83f314ffe408 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:43:25.096042 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1405/2810] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:43:24 2022] DELETE /identity/v3/projects/fdb9dce8fdf141f3a33ae1a8becc6784 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:43:25.101973 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf8433-93d8-4106-930c-2d1ccac4505c None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:25.102257 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf8433-93d8-4106-930c-2d1ccac4505c None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:25.102538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf8433-93d8-4106-930c-2d1ccac4505c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:25.122260 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-8daf8433-93d8-4106-930c-2d1ccac4505c None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:43:25.123568 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-8daf8433-93d8-4106-930c-2d1ccac4505c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:43:25.183636 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1406/2811] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:43:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:43:25.190959 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa44462-e1c6-4bc1-b56b-3691fd1b61b6 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:25.191317 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa44462-e1c6-4bc1-b56b-3691fd1b61b6 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:25.191659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa44462-e1c6-4bc1-b56b-3691fd1b61b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:25.215223 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-0aa44462-e1c6-4bc1-b56b-3691fd1b61b6 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:43:25.216986 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-0aa44462-e1c6-4bc1-b56b-3691fd1b61b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:43:25.225409 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1406/2812] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:43:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:43:25.230089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf8433-93d8-4106-930c-2d1ccac4505c None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:25.240105 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf8433-93d8-4106-930c-2d1ccac4505c None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:25.245072 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f19856-6959-4d96-a0d5-3f4995030f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:25.245702 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f19856-6959-4d96-a0d5-3f4995030f04 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:25.245999 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f19856-6959-4d96-a0d5-3f4995030f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:25.246254 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f19856-6959-4d96-a0d5-3f4995030f04 admin admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:25.247027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f19856-6959-4d96-a0d5-3f4995030f04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:43:25.247332 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f19856-6959-4d96-a0d5-3f4995030f04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:25.248909 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f19856-6959-4d96-a0d5-3f4995030f04 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:25.262132 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1407/2813] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:43:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:43:25.266545 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa44462-e1c6-4bc1-b56b-3691fd1b61b6 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:25.276463 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa44462-e1c6-4bc1-b56b-3691fd1b61b6 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:25.281099 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2148e9-87a0-466c-bb32-d36df5d31f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:25.281691 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2148e9-87a0-466c-bb32-d36df5d31f71 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:25.281969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2148e9-87a0-466c-bb32-d36df5d31f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:25.282237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2148e9-87a0-466c-bb32-d36df5d31f71 admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:25.282903 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2148e9-87a0-466c-bb32-d36df5d31f71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:25.284824 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2148e9-87a0-466c-bb32-d36df5d31f71 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:25.312876 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1407/2814] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:43:25 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:43:25.317093 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f19856-6959-4d96-a0d5-3f4995030f04 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:25.327567 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f19856-6959-4d96-a0d5-3f4995030f04 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:25.385827 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d5e273-5fe1-407c-8c9a-d935747c233e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:25.386427 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5e273-5fe1-407c-8c9a-d935747c233e admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:25.386704 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5e273-5fe1-407c-8c9a-d935747c233e admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:25.387007 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5e273-5fe1-407c-8c9a-d935747c233e admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:25.387672 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d5e273-5fe1-407c-8c9a-d935747c233e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:25.389602 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d5e273-5fe1-407c-8c9a-d935747c233e admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:25.413247 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1408/2815] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:43:25 2022] POST /identity/v3/users => generated 285 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:43:25.417460 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2148e9-87a0-466c-bb32-d36df5d31f71 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:25.427520 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2148e9-87a0-466c-bb32-d36df5d31f71 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:25.432077 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a752f-1ca4-4083-96a1-a3ea231dc887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:25.432669 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a752f-1ca4-4083-96a1-a3ea231dc887 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:25.432947 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a752f-1ca4-4083-96a1-a3ea231dc887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:25.433213 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a752f-1ca4-4083-96a1-a3ea231dc887 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:25.433899 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a752f-1ca4-4083-96a1-a3ea231dc887 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:25.435459 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a752f-1ca4-4083-96a1-a3ea231dc887 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:25.445763 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1408/2816] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:43:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:43:25.449993 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d5e273-5fe1-407c-8c9a-d935747c233e admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:25.460884 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d5e273-5fe1-407c-8c9a-d935747c233e admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:25.465748 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a2edb-21e4-42c6-9e7c-aca0b1fefc53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:25.466396 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a2edb-21e4-42c6-9e7c-aca0b1fefc53 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:25.466651 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a2edb-21e4-42c6-9e7c-aca0b1fefc53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:25.466947 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a2edb-21e4-42c6-9e7c-aca0b1fefc53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:25.467580 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036a2edb-21e4-42c6-9e7c-aca0b1fefc53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:25.469147 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036a2edb-21e4-42c6-9e7c-aca0b1fefc53 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:25.479983 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1409/2817] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:43:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:43:25.484295 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a752f-1ca4-4083-96a1-a3ea231dc887 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:25.494873 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a752f-1ca4-4083-96a1-a3ea231dc887 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:25.499678 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a718cc2-2da5-496f-8b4b-dae2ab435944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:25.500426 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a718cc2-2da5-496f-8b4b-dae2ab435944 admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:25.500774 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a718cc2-2da5-496f-8b4b-dae2ab435944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:25.501114 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a718cc2-2da5-496f-8b4b-dae2ab435944 admin admin] PATH_INFO: `/v3/projects/15719a291c444119b420bc43c3236170/users/e0a2c4ea20774891a1fe01f8faab7e02/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:25.517196 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a718cc2-2da5-496f-8b4b-dae2ab435944 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15719a291c444119b420bc43c3236170, user_id=e0a2c4ea20774891a1fe01f8faab7e02, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:25.519993 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a718cc2-2da5-496f-8b4b-dae2ab435944 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:25.533850 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1409/2818] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:43:25 2022] PUT /identity/v3/projects/15719a291c444119b420bc43c3236170/users/e0a2c4ea20774891a1fe01f8faab7e02/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:43:25.546027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d846549a-98e7-4e1f-87c4-2777632e76d3 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:25.546323 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d846549a-98e7-4e1f-87c4-2777632e76d3 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:25.546595 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d846549a-98e7-4e1f-87c4-2777632e76d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:25.576232 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-d846549a-98e7-4e1f-87c4-2777632e76d3 None None] MFA Rules not processed for user `e0a2c4ea20774891a1fe01f8faab7e02`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:43:25.577601 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-d846549a-98e7-4e1f-87c4-2777632e76d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:43:25.601199 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:43:25.663635 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1410/2819] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:43:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:43:25.681211 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a718cc2-2da5-496f-8b4b-dae2ab435944 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:43:25.683159 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-3a718cc2-2da5-496f-8b4b-dae2ab435944 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:43:25.717891 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:43:25.788073 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a718cc2-2da5-496f-8b4b-dae2ab435944 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:43:25.792212 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336a1cf-90f0-447e-94a6-7df4139c104b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e82b01dbd6e84a9d82baa2c28d77fc28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:43:25.792900 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336a1cf-90f0-447e-94a6-7df4139c104b service glance] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:43:25.793264 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336a1cf-90f0-447e-94a6-7df4139c104b service glance] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:43:25.793602 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336a1cf-90f0-447e-94a6-7df4139c104b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:43:25.803956 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4336a1cf-90f0-447e-94a6-7df4139c104b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:43:25.805617 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4336a1cf-90f0-447e-94a6-7df4139c104b service glance] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:43:25.815410 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1410/2820] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:43:25 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:43:35.659680 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=17 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:43:35.811341 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=26 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:44:15.037215 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d846549a-98e7-4e1f-87c4-2777632e76d3 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:44:15.098460 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d846549a-98e7-4e1f-87c4-2777632e76d3 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:44:15.103051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f14fe-8430-4589-9262-e34398d8d258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:44:15.103687 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f14fe-8430-4589-9262-e34398d8d258 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:44:15.103961 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f14fe-8430-4589-9262-e34398d8d258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:44:15.104229 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f14fe-8430-4589-9262-e34398d8d258 admin admin] PATH_INFO: `/v3/users/e0a2c4ea20774891a1fe01f8faab7e02` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:44:15.105327 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f14fe-8430-4589-9262-e34398d8d258 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0a2c4ea20774891a1fe01f8faab7e02)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:44:15.107929 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f14fe-8430-4589-9262-e34398d8d258 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:44:15.152523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d68f14fe-8430-4589-9262-e34398d8d258 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0a2c4ea20774891a1fe01f8faab7e02'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:44:15.153652 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d68f14fe-8430-4589-9262-e34398d8d258 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0a2c4ea20774891a1fe01f8faab7e02'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:44:15.154342 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d68f14fe-8430-4589-9262-e34398d8d258 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0a2c4ea20774891a1fe01f8faab7e02'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:44:15.166333 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d68f14fe-8430-4589-9262-e34398d8d258 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0a2c4ea20774891a1fe01f8faab7e02'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:44:15.173690 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-d68f14fe-8430-4589-9262-e34398d8d258 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0a2c4ea20774891a1fe01f8faab7e02'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:44:15.186074 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1411/2821] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:44:15 2022] DELETE /identity/v3/users/e0a2c4ea20774891a1fe01f8faab7e02 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:44:15.190744 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a80a6a-55c2-46dc-b7f4-59863b1956e2 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:44:15.191325 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a80a6a-55c2-46dc-b7f4-59863b1956e2 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:44:15.191325 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a80a6a-55c2-46dc-b7f4-59863b1956e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:44:15.210440 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-65a80a6a-55c2-46dc-b7f4-59863b1956e2 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:44:15.211660 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-65a80a6a-55c2-46dc-b7f4-59863b1956e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:44:15.269631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1411/2822] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:44:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:44:15.279140 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f14fe-8430-4589-9262-e34398d8d258 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:44:15.280530 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-d68f14fe-8430-4589-9262-e34398d8d258 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:44:15.339237 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f14fe-8430-4589-9262-e34398d8d258 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:44:15.343403 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0542a5-157a-4579-84fb-029a6824b77d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:44:15.344013 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0542a5-157a-4579-84fb-029a6824b77d service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:44:15.344289 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0542a5-157a-4579-84fb-029a6824b77d service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:44:15.344773 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0542a5-157a-4579-84fb-029a6824b77d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:44:15.355015 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0542a5-157a-4579-84fb-029a6824b77d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:44:15.356862 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0542a5-157a-4579-84fb-029a6824b77d service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:44:15.366205 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1412/2823] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:44:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:44:15.630414 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a80a6a-55c2-46dc-b7f4-59863b1956e2 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:44:15.631943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-65a80a6a-55c2-46dc-b7f4-59863b1956e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:44:15.645864 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a80a6a-55c2-46dc-b7f4-59863b1956e2 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:44:15.651060 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55043dae-9908-4597-a3d8-b9157767f596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:44:15.651661 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55043dae-9908-4597-a3d8-b9157767f596 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:44:15.651924 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55043dae-9908-4597-a3d8-b9157767f596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:44:15.652898 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55043dae-9908-4597-a3d8-b9157767f596 admin admin] PATH_INFO: `/v3/projects/15719a291c444119b420bc43c3236170` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:44:15.653453 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55043dae-9908-4597-a3d8-b9157767f596 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15719a291c444119b420bc43c3236170)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:44:15.655388 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55043dae-9908-4597-a3d8-b9157767f596 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:44:15.701906 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-55043dae-9908-4597-a3d8-b9157767f596 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15719a291c444119b420bc43c3236170', 'request_id': 'req-55043dae-9908-4597-a3d8-b9157767f596'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:44:15.708392 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-55043dae-9908-4597-a3d8-b9157767f596 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15719a291c444119b420bc43c3236170', 'request_id': 'req-55043dae-9908-4597-a3d8-b9157767f596'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:44:15.714588 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-55043dae-9908-4597-a3d8-b9157767f596 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15719a291c444119b420bc43c3236170', 'request_id': 'req-55043dae-9908-4597-a3d8-b9157767f596'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:44:15.728800 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-55043dae-9908-4597-a3d8-b9157767f596 admin admin] The token cache is being invalidate because project 15719a291c444119b420bc43c3236170 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:44:15.729084 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-55043dae-9908-4597-a3d8-b9157767f596 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:44:15.729788 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-55043dae-9908-4597-a3d8-b9157767f596 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:44:15.731342 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1412/2824] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:44:15 2022] DELETE /identity/v3/projects/15719a291c444119b420bc43c3236170 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:44:15.736410 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee9754-7811-4fd0-93a9-1629589bba7b None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:44:15.736730 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee9754-7811-4fd0-93a9-1629589bba7b None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:44:15.737025 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee9754-7811-4fd0-93a9-1629589bba7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:44:15.756908 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-7bee9754-7811-4fd0-93a9-1629589bba7b None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:44:15.758117 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-7bee9754-7811-4fd0-93a9-1629589bba7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:44:15.775051 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:44:15.822939 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1413/2825] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:44:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:44:15.831072 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db667f-c83d-46a5-a57d-c3369a865101 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:44:15.831430 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db667f-c83d-46a5-a57d-c3369a865101 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:44:15.831771 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db667f-c83d-46a5-a57d-c3369a865101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:44:15.851430 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-69db667f-c83d-46a5-a57d-c3369a865101 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:44:15.852623 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-69db667f-c83d-46a5-a57d-c3369a865101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:44:15.860444 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1413/2826] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:44:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:44:15.864740 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bee9754-7811-4fd0-93a9-1629589bba7b None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:44:15.874858 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bee9754-7811-4fd0-93a9-1629589bba7b None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:44:15.879329 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dc0406-5650-49fd-b396-3263f0c3bc41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:44:15.879949 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc0406-5650-49fd-b396-3263f0c3bc41 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:44:15.880227 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc0406-5650-49fd-b396-3263f0c3bc41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:44:15.880499 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc0406-5650-49fd-b396-3263f0c3bc41 admin admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:44:15.881146 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dc0406-5650-49fd-b396-3263f0c3bc41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:44:15.881447 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dc0406-5650-49fd-b396-3263f0c3bc41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:44:15.883059 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dc0406-5650-49fd-b396-3263f0c3bc41 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:44:15.895244 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1414/2827] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:44:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:44:15.899419 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db667f-c83d-46a5-a57d-c3369a865101 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:44:15.908467 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db667f-c83d-46a5-a57d-c3369a865101 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:44:15.913240 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d910a121-5be9-48f2-bddb-f5536e4788b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:44:15.913830 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910a121-5be9-48f2-bddb-f5536e4788b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:44:15.914105 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910a121-5be9-48f2-bddb-f5536e4788b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:44:15.914373 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910a121-5be9-48f2-bddb-f5536e4788b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:44:15.915056 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d910a121-5be9-48f2-bddb-f5536e4788b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:44:15.916925 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d910a121-5be9-48f2-bddb-f5536e4788b3 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:44:15.945176 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1414/2828] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:44:15 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:44:15.948352 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dc0406-5650-49fd-b396-3263f0c3bc41 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:44:15.958462 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dc0406-5650-49fd-b396-3263f0c3bc41 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:44:16.015453 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a7403b-9715-43ef-922e-9b41cfa2466b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:44:16.016064 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7403b-9715-43ef-922e-9b41cfa2466b admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:44:16.016337 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7403b-9715-43ef-922e-9b41cfa2466b admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:44:16.016606 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7403b-9715-43ef-922e-9b41cfa2466b admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:44:16.017230 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a7403b-9715-43ef-922e-9b41cfa2466b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:44:16.019188 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a7403b-9715-43ef-922e-9b41cfa2466b admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:44:16.040135 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1415/2829] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:44:15 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:44:16.044735 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d910a121-5be9-48f2-bddb-f5536e4788b3 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:44:16.054897 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d910a121-5be9-48f2-bddb-f5536e4788b3 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:44:16.059994 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a924177-d368-46dc-95bf-ddb58d3e30e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:44:16.060642 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a924177-d368-46dc-95bf-ddb58d3e30e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:44:16.060918 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a924177-d368-46dc-95bf-ddb58d3e30e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:44:16.061188 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a924177-d368-46dc-95bf-ddb58d3e30e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:44:16.061823 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a924177-d368-46dc-95bf-ddb58d3e30e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:44:16.063427 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a924177-d368-46dc-95bf-ddb58d3e30e9 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:44:16.073090 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1415/2830] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:44:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:44:16.076841 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a7403b-9715-43ef-922e-9b41cfa2466b admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:44:16.086660 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a7403b-9715-43ef-922e-9b41cfa2466b admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:44:16.091314 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ef287-af55-4027-ab90-221c0d3f93c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:44:16.091899 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ef287-af55-4027-ab90-221c0d3f93c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:44:16.092181 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ef287-af55-4027-ab90-221c0d3f93c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:44:16.092457 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ef287-af55-4027-ab90-221c0d3f93c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:44:16.093095 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551ef287-af55-4027-ab90-221c0d3f93c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:44:16.094676 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551ef287-af55-4027-ab90-221c0d3f93c9 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:44:16.105144 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1416/2831] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:44:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 14:44:16.109539 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a924177-d368-46dc-95bf-ddb58d3e30e9 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:44:16.119306 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a924177-d368-46dc-95bf-ddb58d3e30e9 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:44:16.124393 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863e918-0e10-4afb-98e4-9b0990e3a3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:44:16.125077 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863e918-0e10-4afb-98e4-9b0990e3a3f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:44:16.125330 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863e918-0e10-4afb-98e4-9b0990e3a3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:44:16.125627 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863e918-0e10-4afb-98e4-9b0990e3a3f0 admin admin] PATH_INFO: `/v3/projects/043a1697da3f4490ae09be292ed7dc02/users/34414d441c1240899974009763eac672/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:44:16.139873 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7863e918-0e10-4afb-98e4-9b0990e3a3f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=043a1697da3f4490ae09be292ed7dc02, user_id=34414d441c1240899974009763eac672, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:44:16.142487 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7863e918-0e10-4afb-98e4-9b0990e3a3f0 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:44:16.155570 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1416/2832] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:44:16 2022] PUT /identity/v3/projects/043a1697da3f4490ae09be292ed7dc02/users/34414d441c1240899974009763eac672/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:44:16.165915 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775f4a7-78a2-40e8-94c9-817693112a47 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:44:16.166204 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775f4a7-78a2-40e8-94c9-817693112a47 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:44:16.166477 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775f4a7-78a2-40e8-94c9-817693112a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:44:16.195395 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-d775f4a7-78a2-40e8-94c9-817693112a47 None None] MFA Rules not processed for user `34414d441c1240899974009763eac672`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:44:16.196595 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-d775f4a7-78a2-40e8-94c9-817693112a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:44:16.259481 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1417/2833] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:44:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:44:16.270499 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863e918-0e10-4afb-98e4-9b0990e3a3f0 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:44:16.272206 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-7863e918-0e10-4afb-98e4-9b0990e3a3f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:44:16.293487 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:44:16.329707 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863e918-0e10-4afb-98e4-9b0990e3a3f0 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:44:16.333268 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e3a5ce-0a7d-4c41-85d9-7c33fb055f3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:44:16.333857 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e3a5ce-0a7d-4c41-85d9-7c33fb055f3c service cinder] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:44:16.334130 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e3a5ce-0a7d-4c41-85d9-7c33fb055f3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:44:16.334399 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e3a5ce-0a7d-4c41-85d9-7c33fb055f3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:44:16.343046 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e3a5ce-0a7d-4c41-85d9-7c33fb055f3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:44:16.344463 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e3a5ce-0a7d-4c41-85d9-7c33fb055f3c service cinder] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:44:16.352939 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1417/2834] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:44:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:44:18.898341 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775f4a7-78a2-40e8-94c9-817693112a47 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:44:18.917998 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:44:18.962363 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775f4a7-78a2-40e8-94c9-817693112a47 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:44:18.967543 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f5c08-30ae-4315-80c7-6b7e416c48dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:44:18.968166 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f5c08-30ae-4315-80c7-6b7e416c48dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:44:18.968449 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f5c08-30ae-4315-80c7-6b7e416c48dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:44:18.968721 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f5c08-30ae-4315-80c7-6b7e416c48dd admin admin] PATH_INFO: `/v3/users/34414d441c1240899974009763eac672` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:44:18.969826 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15f5c08-30ae-4315-80c7-6b7e416c48dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=34414d441c1240899974009763eac672)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:44:18.971774 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15f5c08-30ae-4315-80c7-6b7e416c48dd admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:44:19.015964 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-f15f5c08-30ae-4315-80c7-6b7e416c48dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34414d441c1240899974009763eac672'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:44:19.016575 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-f15f5c08-30ae-4315-80c7-6b7e416c48dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34414d441c1240899974009763eac672'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:44:19.017165 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-f15f5c08-30ae-4315-80c7-6b7e416c48dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34414d441c1240899974009763eac672'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:44:19.029259 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-f15f5c08-30ae-4315-80c7-6b7e416c48dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34414d441c1240899974009763eac672'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:44:19.035957 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-f15f5c08-30ae-4315-80c7-6b7e416c48dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34414d441c1240899974009763eac672'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:44:19.048366 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1418/2835] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:44:18 2022] DELETE /identity/v3/users/34414d441c1240899974009763eac672 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:44:19.052929 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7677618-388f-4690-a6df-2ee411de4b79 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:44:19.053470 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7677618-388f-4690-a6df-2ee411de4b79 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:44:19.053470 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7677618-388f-4690-a6df-2ee411de4b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:44:19.072611 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-b7677618-388f-4690-a6df-2ee411de4b79 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:44:19.073789 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-b7677618-388f-4690-a6df-2ee411de4b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:44:19.129435 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1418/2836] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:44:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:44:19.139529 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f5c08-30ae-4315-80c7-6b7e416c48dd admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:44:19.140934 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-f15f5c08-30ae-4315-80c7-6b7e416c48dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:44:19.199792 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f5c08-30ae-4315-80c7-6b7e416c48dd admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:44:19.203990 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03148d3d-4189-4515-9866-b45dda25b25f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:44:19.204616 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03148d3d-4189-4515-9866-b45dda25b25f service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:44:19.204893 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03148d3d-4189-4515-9866-b45dda25b25f service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:44:19.205162 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03148d3d-4189-4515-9866-b45dda25b25f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:44:19.214256 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03148d3d-4189-4515-9866-b45dda25b25f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:44:19.216465 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03148d3d-4189-4515-9866-b45dda25b25f service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:44:19.225702 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1419/2837] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:44:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:44:19.486187 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7677618-388f-4690-a6df-2ee411de4b79 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:44:19.487756 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-b7677618-388f-4690-a6df-2ee411de4b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:44:19.504243 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7677618-388f-4690-a6df-2ee411de4b79 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:44:19.509598 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436731ff-a7fc-4770-8e2c-0df0678839ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:44:19.510226 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436731ff-a7fc-4770-8e2c-0df0678839ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:44:19.510498 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436731ff-a7fc-4770-8e2c-0df0678839ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:44:19.510776 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436731ff-a7fc-4770-8e2c-0df0678839ab admin admin] PATH_INFO: `/v3/projects/043a1697da3f4490ae09be292ed7dc02` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:44:19.511881 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436731ff-a7fc-4770-8e2c-0df0678839ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=043a1697da3f4490ae09be292ed7dc02)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:44:19.514458 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436731ff-a7fc-4770-8e2c-0df0678839ab admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:44:19.560880 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-436731ff-a7fc-4770-8e2c-0df0678839ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '043a1697da3f4490ae09be292ed7dc02', 'request_id': 'req-436731ff-a7fc-4770-8e2c-0df0678839ab'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:44:19.567136 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-436731ff-a7fc-4770-8e2c-0df0678839ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '043a1697da3f4490ae09be292ed7dc02', 'request_id': 'req-436731ff-a7fc-4770-8e2c-0df0678839ab'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:44:19.572927 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-436731ff-a7fc-4770-8e2c-0df0678839ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '043a1697da3f4490ae09be292ed7dc02', 'request_id': 'req-436731ff-a7fc-4770-8e2c-0df0678839ab'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:44:19.586232 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-436731ff-a7fc-4770-8e2c-0df0678839ab admin admin] The token cache is being invalidate because project 043a1697da3f4490ae09be292ed7dc02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:44:19.586518 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-436731ff-a7fc-4770-8e2c-0df0678839ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:44:19.587161 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-436731ff-a7fc-4770-8e2c-0df0678839ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:44:19.588886 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1419/2838] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:44:19 2022] DELETE /identity/v3/projects/043a1697da3f4490ae09be292ed7dc02 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:44:29.221569 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=71,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:44:29.583229 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:44:54.919476 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03148d3d-4189-4515-9866-b45dda25b25f service neutron] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:44:54.937350 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03148d3d-4189-4515-9866-b45dda25b25f service neutron] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:44:54.996841 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1877c2-529c-4c2d-a373-f528cdf4ca7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:44:54.998989 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1877c2-529c-4c2d-a373-f528cdf4ca7f service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:44:54.998989 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1877c2-529c-4c2d-a373-f528cdf4ca7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:44:54.998989 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1877c2-529c-4c2d-a373-f528cdf4ca7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:44:54.999811 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-df1877c2-529c-4c2d-a373-f528cdf4ca7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:44:55.089461 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1877c2-529c-4c2d-a373-f528cdf4ca7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:44:55.090984 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1877c2-529c-4c2d-a373-f528cdf4ca7f service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:44:55.101192 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1420/2839] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:44:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:44:57.886431 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436731ff-a7fc-4770-8e2c-0df0678839ab admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:44:57.888397 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-436731ff-a7fc-4770-8e2c-0df0678839ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:44:58.055526 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436731ff-a7fc-4770-8e2c-0df0678839ab admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:44:58.064553 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00910209-5c74-4376-acc0-14d65c224edb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18fe384ecd454fb298dcc788f41dd44b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:44:58.065162 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00910209-5c74-4376-acc0-14d65c224edb service placement] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:44:58.065569 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00910209-5c74-4376-acc0-14d65c224edb service placement] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:44:58.065849 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00910209-5c74-4376-acc0-14d65c224edb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:44:58.067642 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-00910209-5c74-4376-acc0-14d65c224edb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:44:58.079972 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00910209-5c74-4376-acc0-14d65c224edb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:44:58.081542 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00910209-5c74-4376-acc0-14d65c224edb service placement] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:44:58.091616 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1420/2840] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:44:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:45:00.999170 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1877c2-529c-4c2d-a373-f528cdf4ca7f service neutron] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:01.019895 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1877c2-529c-4c2d-a373-f528cdf4ca7f service neutron] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:01.026548 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c2369-5180-474a-bbb9-5b72050ed61f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18fe384ecd454fb298dcc788f41dd44b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:01.026548 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c2369-5180-474a-bbb9-5b72050ed61f service placement] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:01.026548 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c2369-5180-474a-bbb9-5b72050ed61f service placement] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:01.026548 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c2369-5180-474a-bbb9-5b72050ed61f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:01.028483 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-0a1c2369-5180-474a-bbb9-5b72050ed61f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:45:01.044483 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00910209-5c74-4376-acc0-14d65c224edb service placement] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:01.045184 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1c2369-5180-474a-bbb9-5b72050ed61f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:01.046531 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-00910209-5c74-4376-acc0-14d65c224edb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:45:01.047825 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1c2369-5180-474a-bbb9-5b72050ed61f service placement] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:01.059018 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1421/2841] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:45:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 14:45:01.113824 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00910209-5c74-4376-acc0-14d65c224edb service placement] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:01.118510 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792a3f5a-449f-41d2-97f7-c29e54edd949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:01.119176 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a3f5a-449f-41d2-97f7-c29e54edd949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:01.119452 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a3f5a-449f-41d2-97f7-c29e54edd949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:01.119722 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a3f5a-449f-41d2-97f7-c29e54edd949 admin admin] PATH_INFO: `/v3/users/ffe9a464991e4965a0638d0c23b17ae7` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:01.120885 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792a3f5a-449f-41d2-97f7-c29e54edd949 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffe9a464991e4965a0638d0c23b17ae7)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:01.122973 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792a3f5a-449f-41d2-97f7-c29e54edd949 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:01.177544 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-792a3f5a-449f-41d2-97f7-c29e54edd949 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffe9a464991e4965a0638d0c23b17ae7'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:45:01.178202 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-792a3f5a-449f-41d2-97f7-c29e54edd949 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffe9a464991e4965a0638d0c23b17ae7'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:45:01.185543 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-792a3f5a-449f-41d2-97f7-c29e54edd949 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffe9a464991e4965a0638d0c23b17ae7'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:45:01.186179 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-792a3f5a-449f-41d2-97f7-c29e54edd949 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffe9a464991e4965a0638d0c23b17ae7'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:45:01.197079 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:45:01.198651 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-792a3f5a-449f-41d2-97f7-c29e54edd949 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffe9a464991e4965a0638d0c23b17ae7'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:45:01.213083 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1421/2842] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:45:01 2022] DELETE /identity/v3/users/ffe9a464991e4965a0638d0c23b17ae7 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:45:01.380752 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c2369-5180-474a-bbb9-5b72050ed61f service placement] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:01.392952 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c2369-5180-474a-bbb9-5b72050ed61f service placement] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:01.437947 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=100 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:45:01.449849 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0f6cf3-8cc5-4494-9cb5-83600e213e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:01.450480 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f6cf3-8cc5-4494-9cb5-83600e213e03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:01.450758 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f6cf3-8cc5-4494-9cb5-83600e213e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:01.451061 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f6cf3-8cc5-4494-9cb5-83600e213e03 admin admin] PATH_INFO: `/v3/projects/f69d85fa8dc545efb55a0b2949e9f06c` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:01.452188 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0f6cf3-8cc5-4494-9cb5-83600e213e03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f69d85fa8dc545efb55a0b2949e9f06c)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:01.454618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0f6cf3-8cc5-4494-9cb5-83600e213e03 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:01.498670 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-6f0f6cf3-8cc5-4494-9cb5-83600e213e03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f69d85fa8dc545efb55a0b2949e9f06c', 'request_id': 'req-6f0f6cf3-8cc5-4494-9cb5-83600e213e03'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:45:01.505159 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-6f0f6cf3-8cc5-4494-9cb5-83600e213e03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f69d85fa8dc545efb55a0b2949e9f06c', 'request_id': 'req-6f0f6cf3-8cc5-4494-9cb5-83600e213e03'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:45:01.510643 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-6f0f6cf3-8cc5-4494-9cb5-83600e213e03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f69d85fa8dc545efb55a0b2949e9f06c', 'request_id': 'req-6f0f6cf3-8cc5-4494-9cb5-83600e213e03'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:45:01.522919 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-6f0f6cf3-8cc5-4494-9cb5-83600e213e03 admin admin] The token cache is being invalidate because project f69d85fa8dc545efb55a0b2949e9f06c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:45:01.523204 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-6f0f6cf3-8cc5-4494-9cb5-83600e213e03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:45:01.523810 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-6f0f6cf3-8cc5-4494-9cb5-83600e213e03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:45:01.525260 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1422/2843] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:45:01 2022] DELETE /identity/v3/projects/f69d85fa8dc545efb55a0b2949e9f06c => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:45:01.529235 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792a3f5a-449f-41d2-97f7-c29e54edd949 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:01.530665 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-792a3f5a-449f-41d2-97f7-c29e54edd949 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:45:01.543811 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792a3f5a-449f-41d2-97f7-c29e54edd949 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:01.602275 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1d5a2-a3e8-4073-b0a5-284a21fe36f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:01.602922 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1d5a2-a3e8-4073-b0a5-284a21fe36f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:01.603203 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1d5a2-a3e8-4073-b0a5-284a21fe36f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:01.603477 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1d5a2-a3e8-4073-b0a5-284a21fe36f1 admin admin] PATH_INFO: `/v3/users/07def5ab50ed44e1abf49130288c222d` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:01.604604 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac1d5a2-a3e8-4073-b0a5-284a21fe36f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07def5ab50ed44e1abf49130288c222d)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:01.606518 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac1d5a2-a3e8-4073-b0a5-284a21fe36f1 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:01.653215 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-fac1d5a2-a3e8-4073-b0a5-284a21fe36f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07def5ab50ed44e1abf49130288c222d'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:45:01.653896 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-fac1d5a2-a3e8-4073-b0a5-284a21fe36f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07def5ab50ed44e1abf49130288c222d'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:45:01.660843 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-fac1d5a2-a3e8-4073-b0a5-284a21fe36f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07def5ab50ed44e1abf49130288c222d'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:45:01.661516 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-fac1d5a2-a3e8-4073-b0a5-284a21fe36f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07def5ab50ed44e1abf49130288c222d'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:45:01.674144 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-fac1d5a2-a3e8-4073-b0a5-284a21fe36f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07def5ab50ed44e1abf49130288c222d'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:45:01.686962 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1422/2844] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:45:01 2022] DELETE /identity/v3/users/07def5ab50ed44e1abf49130288c222d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:45:01.862752 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0f6cf3-8cc5-4494-9cb5-83600e213e03 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:01.864230 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-6f0f6cf3-8cc5-4494-9cb5-83600e213e03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:45:01.877215 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0f6cf3-8cc5-4494-9cb5-83600e213e03 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:01.935398 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97184c13-bd65-4174-80cf-44764939072e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:01.935951 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97184c13-bd65-4174-80cf-44764939072e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:01.936223 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97184c13-bd65-4174-80cf-44764939072e admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:01.936492 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97184c13-bd65-4174-80cf-44764939072e admin admin] PATH_INFO: `/v3/projects/4039b94848584573b523d0c3d844c47d` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:01.937585 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97184c13-bd65-4174-80cf-44764939072e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4039b94848584573b523d0c3d844c47d)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:01.939513 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97184c13-bd65-4174-80cf-44764939072e admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:01.985540 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-97184c13-bd65-4174-80cf-44764939072e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4039b94848584573b523d0c3d844c47d', 'request_id': 'req-97184c13-bd65-4174-80cf-44764939072e'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:45:01.991597 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-97184c13-bd65-4174-80cf-44764939072e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4039b94848584573b523d0c3d844c47d', 'request_id': 'req-97184c13-bd65-4174-80cf-44764939072e'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:45:01.997225 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-97184c13-bd65-4174-80cf-44764939072e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4039b94848584573b523d0c3d844c47d', 'request_id': 'req-97184c13-bd65-4174-80cf-44764939072e'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:45:02.009902 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-97184c13-bd65-4174-80cf-44764939072e admin admin] The token cache is being invalidate because project 4039b94848584573b523d0c3d844c47d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:45:02.010508 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-97184c13-bd65-4174-80cf-44764939072e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:45:02.010837 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-97184c13-bd65-4174-80cf-44764939072e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:45:02.012337 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1423/2845] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:45:01 2022] DELETE /identity/v3/projects/4039b94848584573b523d0c3d844c47d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:45:10.351552 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1d5a2-a3e8-4073-b0a5-284a21fe36f1 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:10.353120 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-fac1d5a2-a3e8-4073-b0a5-284a21fe36f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:45:10.421326 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:45:10.456740 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1d5a2-a3e8-4073-b0a5-284a21fe36f1 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:10.517549 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166ad0e-47ec-4933-9c77-9a7c48091948 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:10.518149 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166ad0e-47ec-4933-9c77-9a7c48091948 service nova] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:10.518424 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166ad0e-47ec-4933-9c77-9a7c48091948 service nova] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:10.518692 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166ad0e-47ec-4933-9c77-9a7c48091948 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:10.520442 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-4166ad0e-47ec-4933-9c77-9a7c48091948 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:45:10.576055 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4166ad0e-47ec-4933-9c77-9a7c48091948 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:10.578203 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4166ad0e-47ec-4933-9c77-9a7c48091948 service nova] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:10.586881 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1423/2846] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:45:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:45:12.006914 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=80,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:45:20.582981 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=65 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:45:29.754456 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97184c13-bd65-4174-80cf-44764939072e admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:29.756084 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-97184c13-bd65-4174-80cf-44764939072e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:45:29.832059 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97184c13-bd65-4174-80cf-44764939072e admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:29.837248 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9549d6f-d478-4432-9dae-5444e392c560 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18fe384ecd454fb298dcc788f41dd44b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:29.838279 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9549d6f-d478-4432-9dae-5444e392c560 service placement] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:29.838279 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9549d6f-d478-4432-9dae-5444e392c560 service placement] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:29.838586 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9549d6f-d478-4432-9dae-5444e392c560 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:29.840121 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-f9549d6f-d478-4432-9dae-5444e392c560 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:45:29.852485 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9549d6f-d478-4432-9dae-5444e392c560 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:29.854231 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9549d6f-d478-4432-9dae-5444e392c560 service placement] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:29.865318 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1424/2847] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:45:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:45:39.860284 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=32 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:45:41.358674 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166ad0e-47ec-4933-9c77-9a7c48091948 service nova] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:41.360089 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-4166ad0e-47ec-4933-9c77-9a7c48091948 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:45:41.435943 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166ad0e-47ec-4933-9c77-9a7c48091948 service nova] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:41.439629 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1fa59e-51f5-417d-8f6b-0545f5d2fcf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:41.440265 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1fa59e-51f5-417d-8f6b-0545f5d2fcf1 service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:41.440522 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1fa59e-51f5-417d-8f6b-0545f5d2fcf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:41.440789 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1fa59e-51f5-417d-8f6b-0545f5d2fcf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:41.442465 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-7c1fa59e-51f5-417d-8f6b-0545f5d2fcf1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:45:41.502258 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1fa59e-51f5-417d-8f6b-0545f5d2fcf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:41.503704 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1fa59e-51f5-417d-8f6b-0545f5d2fcf1 service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:41.513063 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1424/2848] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:45:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:45:44.574912 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9549d6f-d478-4432-9dae-5444e392c560 service placement] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:44.576443 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-f9549d6f-d478-4432-9dae-5444e392c560 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:45:44.595470 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9549d6f-d478-4432-9dae-5444e392c560 service placement] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:44.602276 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3b5b04-dc27-4420-8b21-16c9d725afd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:44.602984 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b5b04-dc27-4420-8b21-16c9d725afd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:44.603399 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b5b04-dc27-4420-8b21-16c9d725afd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:44.603675 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b5b04-dc27-4420-8b21-16c9d725afd9 admin admin] PATH_INFO: `/v3/users/f5d7b40e78a049cd83d079291c3601ff` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:44.604846 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3b5b04-dc27-4420-8b21-16c9d725afd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5d7b40e78a049cd83d079291c3601ff)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:44.606944 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3b5b04-dc27-4420-8b21-16c9d725afd9 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:44.667778 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9c3b5b04-dc27-4420-8b21-16c9d725afd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5d7b40e78a049cd83d079291c3601ff'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:45:44.668666 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9c3b5b04-dc27-4420-8b21-16c9d725afd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5d7b40e78a049cd83d079291c3601ff'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:45:44.669497 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9c3b5b04-dc27-4420-8b21-16c9d725afd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5d7b40e78a049cd83d079291c3601ff'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:45:44.682947 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9c3b5b04-dc27-4420-8b21-16c9d725afd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5d7b40e78a049cd83d079291c3601ff'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:45:44.691206 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-9c3b5b04-dc27-4420-8b21-16c9d725afd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5d7b40e78a049cd83d079291c3601ff'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:45:44.704565 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1425/2849] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:45:44 2022] DELETE /identity/v3/users/f5d7b40e78a049cd83d079291c3601ff => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:45:44.876041 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1fa59e-51f5-417d-8f6b-0545f5d2fcf1 service neutron] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:44.887410 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1fa59e-51f5-417d-8f6b-0545f5d2fcf1 service neutron] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:44.944038 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65200228-0ba7-4dc4-ad44-23fdb19621a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:44.944614 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65200228-0ba7-4dc4-ad44-23fdb19621a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:44.944890 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65200228-0ba7-4dc4-ad44-23fdb19621a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:44.945174 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65200228-0ba7-4dc4-ad44-23fdb19621a2 admin admin] PATH_INFO: `/v3/projects/cfc1e3a42706435a93301d76ee7b74ed` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:44.946249 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65200228-0ba7-4dc4-ad44-23fdb19621a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfc1e3a42706435a93301d76ee7b74ed)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:44.948169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65200228-0ba7-4dc4-ad44-23fdb19621a2 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:44.982623 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:45:44.996688 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-65200228-0ba7-4dc4-ad44-23fdb19621a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfc1e3a42706435a93301d76ee7b74ed', 'request_id': 'req-65200228-0ba7-4dc4-ad44-23fdb19621a2'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:45:45.003340 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-65200228-0ba7-4dc4-ad44-23fdb19621a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfc1e3a42706435a93301d76ee7b74ed', 'request_id': 'req-65200228-0ba7-4dc4-ad44-23fdb19621a2'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:45:45.008810 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-65200228-0ba7-4dc4-ad44-23fdb19621a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfc1e3a42706435a93301d76ee7b74ed', 'request_id': 'req-65200228-0ba7-4dc4-ad44-23fdb19621a2'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:45:45.020309 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-65200228-0ba7-4dc4-ad44-23fdb19621a2 admin admin] The token cache is being invalidate because project cfc1e3a42706435a93301d76ee7b74ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:45:45.020638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-65200228-0ba7-4dc4-ad44-23fdb19621a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:45:45.021234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-65200228-0ba7-4dc4-ad44-23fdb19621a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:45:45.022766 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1425/2850] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:45:44 2022] DELETE /identity/v3/projects/cfc1e3a42706435a93301d76ee7b74ed => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:45:45.028150 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab72880-adf0-41f5-b5a9-ecbd5a623027 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:45.028437 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab72880-adf0-41f5-b5a9-ecbd5a623027 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:45.028713 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab72880-adf0-41f5-b5a9-ecbd5a623027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:45.086621 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-0ab72880-adf0-41f5-b5a9-ecbd5a623027 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:45:45.087836 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-0ab72880-adf0-41f5-b5a9-ecbd5a623027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:45:45.145784 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1426/2851] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:45:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:45:45.153689 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98c061-e673-4d12-8689-78008d8c29a4 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:45.153974 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98c061-e673-4d12-8689-78008d8c29a4 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:45.154267 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98c061-e673-4d12-8689-78008d8c29a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:45.173548 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-dd98c061-e673-4d12-8689-78008d8c29a4 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:45:45.175324 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-dd98c061-e673-4d12-8689-78008d8c29a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:45:45.183095 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1426/2852] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:45:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:45:45.187168 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab72880-adf0-41f5-b5a9-ecbd5a623027 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:45.197194 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab72880-adf0-41f5-b5a9-ecbd5a623027 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:45.201653 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca4752-c1d6-4ed7-985f-2a9ed00b2cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:45.202241 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca4752-c1d6-4ed7-985f-2a9ed00b2cdc admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:45.202519 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca4752-c1d6-4ed7-985f-2a9ed00b2cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:45.202790 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca4752-c1d6-4ed7-985f-2a9ed00b2cdc admin admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:45.203454 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca4752-c1d6-4ed7-985f-2a9ed00b2cdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:45:45.203754 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca4752-c1d6-4ed7-985f-2a9ed00b2cdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:45.205320 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca4752-c1d6-4ed7-985f-2a9ed00b2cdc admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:45.217265 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1427/2853] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:45:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:45:45.221234 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd98c061-e673-4d12-8689-78008d8c29a4 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:45.230919 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd98c061-e673-4d12-8689-78008d8c29a4 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:45.235937 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d6f1bb-3445-45a4-a33c-c2e230acd4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:45.236545 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d6f1bb-3445-45a4-a33c-c2e230acd4f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:45.236820 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d6f1bb-3445-45a4-a33c-c2e230acd4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:45.237087 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d6f1bb-3445-45a4-a33c-c2e230acd4f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:45.237755 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d6f1bb-3445-45a4-a33c-c2e230acd4f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:45.239716 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d6f1bb-3445-45a4-a33c-c2e230acd4f0 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:45.273774 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1427/2854] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:45:45 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:45:45.278748 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca4752-c1d6-4ed7-985f-2a9ed00b2cdc admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:45.289891 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca4752-c1d6-4ed7-985f-2a9ed00b2cdc admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:45.315378 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:45:45.346306 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844d5cc6-b78b-446b-a519-0a005cb3ad4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:45.347487 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d5cc6-b78b-446b-a519-0a005cb3ad4e admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:45.347765 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d5cc6-b78b-446b-a519-0a005cb3ad4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:45.348035 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d5cc6-b78b-446b-a519-0a005cb3ad4e admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:45.348666 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844d5cc6-b78b-446b-a519-0a005cb3ad4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:45.350581 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844d5cc6-b78b-446b-a519-0a005cb3ad4e admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:45.371371 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1428/2855] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:45:45 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 14:45:45.375604 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d6f1bb-3445-45a4-a33c-c2e230acd4f0 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:45.386177 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d6f1bb-3445-45a4-a33c-c2e230acd4f0 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:45.390738 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531a288-0091-456b-9850-5dfd8aa00d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:45.391359 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531a288-0091-456b-9850-5dfd8aa00d9f admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:45.391640 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531a288-0091-456b-9850-5dfd8aa00d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:45.391909 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531a288-0091-456b-9850-5dfd8aa00d9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:45.392542 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531a288-0091-456b-9850-5dfd8aa00d9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:45.394695 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531a288-0091-456b-9850-5dfd8aa00d9f admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:45.404410 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1428/2856] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:45:45.408167 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844d5cc6-b78b-446b-a519-0a005cb3ad4e admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:45.418097 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844d5cc6-b78b-446b-a519-0a005cb3ad4e admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:45.422598 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5573eef2-347d-4138-a16a-62b1db7599ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:45.423211 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5573eef2-347d-4138-a16a-62b1db7599ff admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:45.423489 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5573eef2-347d-4138-a16a-62b1db7599ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:45.423763 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5573eef2-347d-4138-a16a-62b1db7599ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:45.424387 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5573eef2-347d-4138-a16a-62b1db7599ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:45.425947 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5573eef2-347d-4138-a16a-62b1db7599ff admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:45.436508 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1429/2857] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:45:45.441020 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531a288-0091-456b-9850-5dfd8aa00d9f admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:45.450866 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531a288-0091-456b-9850-5dfd8aa00d9f admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:45.455468 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487cfec-6416-4914-bd55-25ea3c572bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:45.456714 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487cfec-6416-4914-bd55-25ea3c572bdc admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:45.456994 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487cfec-6416-4914-bd55-25ea3c572bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:45.457265 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487cfec-6416-4914-bd55-25ea3c572bdc admin admin] PATH_INFO: `/v3/projects/1b760bf5beae47c5a9f75e921f8c55eb/users/58f8b828dbc14525ba70827291e35057/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:45.471406 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2487cfec-6416-4914-bd55-25ea3c572bdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b760bf5beae47c5a9f75e921f8c55eb, user_id=58f8b828dbc14525ba70827291e35057, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:45.474008 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2487cfec-6416-4914-bd55-25ea3c572bdc admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:45.486537 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1429/2858] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:45:45 2022] PUT /identity/v3/projects/1b760bf5beae47c5a9f75e921f8c55eb/users/58f8b828dbc14525ba70827291e35057/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:45:45.497981 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b53a4-733a-4434-8ab7-09f98fc5e503 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:45.498268 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b53a4-733a-4434-8ab7-09f98fc5e503 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:45.498542 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b53a4-733a-4434-8ab7-09f98fc5e503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:45.526183 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-5d0b53a4-733a-4434-8ab7-09f98fc5e503 None None] MFA Rules not processed for user `58f8b828dbc14525ba70827291e35057`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:45:45.527346 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-5d0b53a4-733a-4434-8ab7-09f98fc5e503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:45:45.590201 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1430/2859] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:45:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:45:45.604113 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487cfec-6416-4914-bd55-25ea3c572bdc admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:45.605791 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-2487cfec-6416-4914-bd55-25ea3c572bdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:45:45.681465 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487cfec-6416-4914-bd55-25ea3c572bdc admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:45.686973 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c58276-f95f-4014-bdf9-f5bd307c8361 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79eaaae97aac4282a613939e5b768376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:45.687572 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c58276-f95f-4014-bdf9-f5bd307c8361 service nova] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:45.687848 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c58276-f95f-4014-bdf9-f5bd307c8361 service nova] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:45.688119 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c58276-f95f-4014-bdf9-f5bd307c8361 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:45.695849 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c58276-f95f-4014-bdf9-f5bd307c8361 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:45.697424 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c58276-f95f-4014-bdf9-f5bd307c8361 service nova] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:45.706615 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1430/2860] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:45:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:45:54.056432 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b53a4-733a-4434-8ab7-09f98fc5e503 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:54.117027 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b53a4-733a-4434-8ab7-09f98fc5e503 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:54.121451 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9889b7d-9e95-40cd-a351-2f15d20fba1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:54.122076 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9889b7d-9e95-40cd-a351-2f15d20fba1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:54.122353 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9889b7d-9e95-40cd-a351-2f15d20fba1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:54.122626 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9889b7d-9e95-40cd-a351-2f15d20fba1f admin admin] PATH_INFO: `/v3/users/58f8b828dbc14525ba70827291e35057` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:54.123762 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9889b7d-9e95-40cd-a351-2f15d20fba1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=58f8b828dbc14525ba70827291e35057)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:54.125665 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9889b7d-9e95-40cd-a351-2f15d20fba1f admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:54.165212 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:45:54.172120 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-b9889b7d-9e95-40cd-a351-2f15d20fba1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58f8b828dbc14525ba70827291e35057'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:45:54.172819 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-b9889b7d-9e95-40cd-a351-2f15d20fba1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58f8b828dbc14525ba70827291e35057'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:45:54.173732 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-b9889b7d-9e95-40cd-a351-2f15d20fba1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58f8b828dbc14525ba70827291e35057'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:45:54.185816 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-b9889b7d-9e95-40cd-a351-2f15d20fba1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58f8b828dbc14525ba70827291e35057'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:45:54.192546 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-b9889b7d-9e95-40cd-a351-2f15d20fba1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58f8b828dbc14525ba70827291e35057'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:45:54.204879 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1431/2861] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:45:54 2022] DELETE /identity/v3/users/58f8b828dbc14525ba70827291e35057 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:45:54.209921 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac704a0-ff19-4760-85f0-f2a461a31bec None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:54.210410 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac704a0-ff19-4760-85f0-f2a461a31bec None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:54.210561 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac704a0-ff19-4760-85f0-f2a461a31bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:54.229367 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-7ac704a0-ff19-4760-85f0-f2a461a31bec None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:45:54.230537 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-7ac704a0-ff19-4760-85f0-f2a461a31bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:45:54.267539 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:45:54.291838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1431/2862] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:45:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:45:54.301627 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9889b7d-9e95-40cd-a351-2f15d20fba1f admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:54.303098 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-b9889b7d-9e95-40cd-a351-2f15d20fba1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:45:54.360836 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9889b7d-9e95-40cd-a351-2f15d20fba1f admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:54.364615 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa9848a-9039-4eea-bb0d-1714dc69deee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:54.365214 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9848a-9039-4eea-bb0d-1714dc69deee service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:54.365487 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9848a-9039-4eea-bb0d-1714dc69deee service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:54.365784 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9848a-9039-4eea-bb0d-1714dc69deee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:54.374680 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa9848a-9039-4eea-bb0d-1714dc69deee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:54.376070 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa9848a-9039-4eea-bb0d-1714dc69deee service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:54.386127 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1432/2863] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:45:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:45:54.649991 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac704a0-ff19-4760-85f0-f2a461a31bec None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:54.651481 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-7ac704a0-ff19-4760-85f0-f2a461a31bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:45:54.663816 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac704a0-ff19-4760-85f0-f2a461a31bec None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:54.668789 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d9be42-07af-4090-92ba-b4d4dd3f3ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:54.669428 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9be42-07af-4090-92ba-b4d4dd3f3ec6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:54.669738 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9be42-07af-4090-92ba-b4d4dd3f3ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:54.670010 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9be42-07af-4090-92ba-b4d4dd3f3ec6 admin admin] PATH_INFO: `/v3/projects/1b760bf5beae47c5a9f75e921f8c55eb` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:54.671221 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d9be42-07af-4090-92ba-b4d4dd3f3ec6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b760bf5beae47c5a9f75e921f8c55eb)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:54.673122 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d9be42-07af-4090-92ba-b4d4dd3f3ec6 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:54.721139 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-f4d9be42-07af-4090-92ba-b4d4dd3f3ec6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b760bf5beae47c5a9f75e921f8c55eb', 'request_id': 'req-f4d9be42-07af-4090-92ba-b4d4dd3f3ec6'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:45:54.727989 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-f4d9be42-07af-4090-92ba-b4d4dd3f3ec6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b760bf5beae47c5a9f75e921f8c55eb', 'request_id': 'req-f4d9be42-07af-4090-92ba-b4d4dd3f3ec6'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:45:54.733652 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-f4d9be42-07af-4090-92ba-b4d4dd3f3ec6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b760bf5beae47c5a9f75e921f8c55eb', 'request_id': 'req-f4d9be42-07af-4090-92ba-b4d4dd3f3ec6'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:45:54.745585 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-f4d9be42-07af-4090-92ba-b4d4dd3f3ec6 admin admin] The token cache is being invalidate because project 1b760bf5beae47c5a9f75e921f8c55eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:45:54.745872 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-f4d9be42-07af-4090-92ba-b4d4dd3f3ec6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:45:54.746492 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-f4d9be42-07af-4090-92ba-b4d4dd3f3ec6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:45:54.747987 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1432/2864] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:45:54 2022] DELETE /identity/v3/projects/1b760bf5beae47c5a9f75e921f8c55eb => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:45:54.753012 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71407280-0c0a-4497-a6fb-2ce128908be9 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:54.753293 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71407280-0c0a-4497-a6fb-2ce128908be9 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:54.753564 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71407280-0c0a-4497-a6fb-2ce128908be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:54.774211 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-71407280-0c0a-4497-a6fb-2ce128908be9 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:45:54.775440 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-71407280-0c0a-4497-a6fb-2ce128908be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:45:54.833593 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1433/2865] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:45:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:45:54.841024 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc323db5-a354-48e7-aa47-cbef43741595 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:54.841314 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc323db5-a354-48e7-aa47-cbef43741595 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:54.841595 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc323db5-a354-48e7-aa47-cbef43741595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:54.861492 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-dc323db5-a354-48e7-aa47-cbef43741595 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:45:54.862697 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-dc323db5-a354-48e7-aa47-cbef43741595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:45:54.870575 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1433/2866] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:45:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:45:54.874744 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71407280-0c0a-4497-a6fb-2ce128908be9 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:54.884254 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71407280-0c0a-4497-a6fb-2ce128908be9 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:54.888693 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055b936-aa0f-4f6a-971c-aa057f54f753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:54.889278 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055b936-aa0f-4f6a-971c-aa057f54f753 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:54.889551 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055b936-aa0f-4f6a-971c-aa057f54f753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:54.889819 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055b936-aa0f-4f6a-971c-aa057f54f753 admin admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:54.890454 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8055b936-aa0f-4f6a-971c-aa057f54f753 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:45:54.890776 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8055b936-aa0f-4f6a-971c-aa057f54f753 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:54.892426 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8055b936-aa0f-4f6a-971c-aa057f54f753 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:54.904436 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1434/2867] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:45:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:45:54.908452 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc323db5-a354-48e7-aa47-cbef43741595 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:54.918341 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc323db5-a354-48e7-aa47-cbef43741595 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:54.922917 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4788a-dda2-4d75-9f71-80ce51aa5202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:54.923506 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4788a-dda2-4d75-9f71-80ce51aa5202 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:54.923789 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4788a-dda2-4d75-9f71-80ce51aa5202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:54.924062 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4788a-dda2-4d75-9f71-80ce51aa5202 admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:54.924696 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e4788a-dda2-4d75-9f71-80ce51aa5202 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:54.926588 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e4788a-dda2-4d75-9f71-80ce51aa5202 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:54.953311 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1434/2868] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:45:54 2022] POST /identity/v3/projects => generated 381 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:45:54.957817 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055b936-aa0f-4f6a-971c-aa057f54f753 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:54.967564 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055b936-aa0f-4f6a-971c-aa057f54f753 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:55.009376 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:45:55.023538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133574b-3614-48b1-9af6-328c17f2b3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:55.024127 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133574b-3614-48b1-9af6-328c17f2b3f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:55.024402 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133574b-3614-48b1-9af6-328c17f2b3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:55.024669 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133574b-3614-48b1-9af6-328c17f2b3f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:55.025288 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d133574b-3614-48b1-9af6-328c17f2b3f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:55.027204 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d133574b-3614-48b1-9af6-328c17f2b3f6 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:55.048189 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1435/2869] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:45:54 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:45:55.052320 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4788a-dda2-4d75-9f71-80ce51aa5202 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:55.062433 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4788a-dda2-4d75-9f71-80ce51aa5202 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:55.067079 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7ab2a3-7df2-4c72-b5d9-3cd94a7bceae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:55.067671 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ab2a3-7df2-4c72-b5d9-3cd94a7bceae admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:55.067952 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ab2a3-7df2-4c72-b5d9-3cd94a7bceae admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:55.068222 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ab2a3-7df2-4c72-b5d9-3cd94a7bceae admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:55.068856 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7ab2a3-7df2-4c72-b5d9-3cd94a7bceae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:55.070440 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7ab2a3-7df2-4c72-b5d9-3cd94a7bceae admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:55.080415 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1435/2870] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:45:55.084488 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133574b-3614-48b1-9af6-328c17f2b3f6 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:55.094376 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133574b-3614-48b1-9af6-328c17f2b3f6 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:55.099904 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93462e2-aa67-4fc1-af96-58493b3f8f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:55.100518 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93462e2-aa67-4fc1-af96-58493b3f8f5f admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:55.100793 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93462e2-aa67-4fc1-af96-58493b3f8f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:55.101059 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93462e2-aa67-4fc1-af96-58493b3f8f5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:55.101689 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93462e2-aa67-4fc1-af96-58493b3f8f5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:55.103338 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93462e2-aa67-4fc1-af96-58493b3f8f5f admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:55.113852 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1436/2871] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:45:55.117740 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7ab2a3-7df2-4c72-b5d9-3cd94a7bceae admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:55.127500 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7ab2a3-7df2-4c72-b5d9-3cd94a7bceae admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:55.132516 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781577f8-6179-4682-91eb-f171e92a0502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:55.133144 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781577f8-6179-4682-91eb-f171e92a0502 admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:55.133416 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781577f8-6179-4682-91eb-f171e92a0502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:55.133678 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781577f8-6179-4682-91eb-f171e92a0502 admin admin] PATH_INFO: `/v3/projects/ba7c5a4890d44dc59a7a7df9a7065a24/users/bedf89585ec94eda858e11dd25882810/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:55.148304 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781577f8-6179-4682-91eb-f171e92a0502 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba7c5a4890d44dc59a7a7df9a7065a24, user_id=bedf89585ec94eda858e11dd25882810, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:55.150860 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781577f8-6179-4682-91eb-f171e92a0502 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:55.163020 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1436/2872] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:45:55 2022] PUT /identity/v3/projects/ba7c5a4890d44dc59a7a7df9a7065a24/users/bedf89585ec94eda858e11dd25882810/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:45:55.173358 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ca4a2-dc8c-4132-bc8b-d4d55c51ca8b None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:55.173643 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ca4a2-dc8c-4132-bc8b-d4d55c51ca8b None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:55.173912 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ca4a2-dc8c-4132-bc8b-d4d55c51ca8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:55.201765 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-c32ca4a2-dc8c-4132-bc8b-d4d55c51ca8b None None] MFA Rules not processed for user `bedf89585ec94eda858e11dd25882810`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:45:55.203059 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-c32ca4a2-dc8c-4132-bc8b-d4d55c51ca8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:45:55.265815 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1437/2873] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:45:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:45:55.270953 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781577f8-6179-4682-91eb-f171e92a0502 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:55.332094 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781577f8-6179-4682-91eb-f171e92a0502 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:55.336659 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885857f4-6628-46b3-9fed-cf8e7d157621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:55.337280 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885857f4-6628-46b3-9fed-cf8e7d157621 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:55.337561 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885857f4-6628-46b3-9fed-cf8e7d157621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:55.337862 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885857f4-6628-46b3-9fed-cf8e7d157621 admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:55.338516 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885857f4-6628-46b3-9fed-cf8e7d157621 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:55.340450 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885857f4-6628-46b3-9fed-cf8e7d157621 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:55.345700 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:45:55.369171 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1437/2874] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:45:55 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:45:55.373401 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32ca4a2-dc8c-4132-bc8b-d4d55c51ca8b None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:55.384486 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32ca4a2-dc8c-4132-bc8b-d4d55c51ca8b None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:55.439218 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed6b98-3326-4418-86b5-2e8fd15c1a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:55.439746 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed6b98-3326-4418-86b5-2e8fd15c1a2a admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:55.441568 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed6b98-3326-4418-86b5-2e8fd15c1a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:55.441568 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed6b98-3326-4418-86b5-2e8fd15c1a2a admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:55.441568 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed6b98-3326-4418-86b5-2e8fd15c1a2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:55.443652 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed6b98-3326-4418-86b5-2e8fd15c1a2a admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:55.465684 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1438/2875] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:45:55 2022] POST /identity/v3/users => generated 296 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:45:55.469688 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885857f4-6628-46b3-9fed-cf8e7d157621 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:55.480032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885857f4-6628-46b3-9fed-cf8e7d157621 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:55.484558 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b6851-3eff-4b9f-a313-d7d8972d9728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:55.485148 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b6851-3eff-4b9f-a313-d7d8972d9728 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:55.485427 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b6851-3eff-4b9f-a313-d7d8972d9728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:55.485698 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b6851-3eff-4b9f-a313-d7d8972d9728 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:55.486333 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b6851-3eff-4b9f-a313-d7d8972d9728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:55.488431 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b6851-3eff-4b9f-a313-d7d8972d9728 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:55.498094 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1438/2876] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:45:55.502156 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed6b98-3326-4418-86b5-2e8fd15c1a2a admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:55.512665 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed6b98-3326-4418-86b5-2e8fd15c1a2a admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:55.517211 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decd7b2b-2930-4c43-bf20-80c5474ee330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:55.517853 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd7b2b-2930-4c43-bf20-80c5474ee330 admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:55.518136 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd7b2b-2930-4c43-bf20-80c5474ee330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:55.518414 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd7b2b-2930-4c43-bf20-80c5474ee330 admin admin] PATH_INFO: `/v3/projects/50242463cdf64de89410353cb261e5aa/users/5b15c48d37f84bb890d2199b42dde8bf/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:55.532996 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decd7b2b-2930-4c43-bf20-80c5474ee330 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50242463cdf64de89410353cb261e5aa, user_id=5b15c48d37f84bb890d2199b42dde8bf, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:55.535699 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decd7b2b-2930-4c43-bf20-80c5474ee330 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:55.548677 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1439/2877] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:45:55 2022] PUT /identity/v3/projects/50242463cdf64de89410353cb261e5aa/users/5b15c48d37f84bb890d2199b42dde8bf/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:45:55.552958 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b6851-3eff-4b9f-a313-d7d8972d9728 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:55.562711 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b6851-3eff-4b9f-a313-d7d8972d9728 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:55.622906 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d284e4a-e4c5-4b22-a996-7081ea8ac5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:55.623513 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d284e4a-e4c5-4b22-a996-7081ea8ac5ba admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:55.623790 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d284e4a-e4c5-4b22-a996-7081ea8ac5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:55.624058 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d284e4a-e4c5-4b22-a996-7081ea8ac5ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:55.624693 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d284e4a-e4c5-4b22-a996-7081ea8ac5ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:55.626266 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d284e4a-e4c5-4b22-a996-7081ea8ac5ba admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:55.636577 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1439/2878] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:45:55.640667 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decd7b2b-2930-4c43-bf20-80c5474ee330 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:55.650584 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decd7b2b-2930-4c43-bf20-80c5474ee330 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:55.655362 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96617e7f-5673-49c8-bdaf-fe59eee7b145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:55.655994 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96617e7f-5673-49c8-bdaf-fe59eee7b145 admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:55.656267 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96617e7f-5673-49c8-bdaf-fe59eee7b145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:55.656537 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96617e7f-5673-49c8-bdaf-fe59eee7b145 admin admin] PATH_INFO: `/v3/projects/50242463cdf64de89410353cb261e5aa/users/5b15c48d37f84bb890d2199b42dde8bf/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:55.658410 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96617e7f-5673-49c8-bdaf-fe59eee7b145 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50242463cdf64de89410353cb261e5aa, user_id=5b15c48d37f84bb890d2199b42dde8bf, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:55.661062 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96617e7f-5673-49c8-bdaf-fe59eee7b145 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:55.671352 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1440/2879] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:45:55 2022] PUT /identity/v3/projects/50242463cdf64de89410353cb261e5aa/users/5b15c48d37f84bb890d2199b42dde8bf/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:45:55.683120 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac0150-05d0-49b1-8799-8e01acc81a78 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:55.683404 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac0150-05d0-49b1-8799-8e01acc81a78 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:55.683677 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac0150-05d0-49b1-8799-8e01acc81a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:55.711258 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-d1ac0150-05d0-49b1-8799-8e01acc81a78 None None] MFA Rules not processed for user `5b15c48d37f84bb890d2199b42dde8bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:45:55.712379 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-d1ac0150-05d0-49b1-8799-8e01acc81a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:45:55.782804 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1440/2880] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:45:55 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:45:55.795840 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96617e7f-5673-49c8-bdaf-fe59eee7b145 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:55.797572 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-96617e7f-5673-49c8-bdaf-fe59eee7b145 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:45:55.807897 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:45:55.857439 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96617e7f-5673-49c8-bdaf-fe59eee7b145 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:55.861089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e0ca8c-4722-4dd4-8c2a-df0926bfd3e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:55.861701 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0ca8c-4722-4dd4-8c2a-df0926bfd3e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:55.861979 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0ca8c-4722-4dd4-8c2a-df0926bfd3e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:55.862802 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0ca8c-4722-4dd4-8c2a-df0926bfd3e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:55.872468 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e0ca8c-4722-4dd4-8c2a-df0926bfd3e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:55.873940 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e0ca8c-4722-4dd4-8c2a-df0926bfd3e4 service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:55.883275 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1441/2881] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:45:55 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:45:55.957774 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac0150-05d0-49b1-8799-8e01acc81a78 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:45:55.967297 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac0150-05d0-49b1-8799-8e01acc81a78 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:45:55.970707 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641e02c-f6d3-4607-8ae0-8de6013e1fa4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:45:55.971325 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641e02c-f6d3-4607-8ae0-8de6013e1fa4 service cinder] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:45:55.972087 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641e02c-f6d3-4607-8ae0-8de6013e1fa4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:45:55.972355 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641e02c-f6d3-4607-8ae0-8de6013e1fa4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:45:56.025407 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:45:56.027926 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b641e02c-f6d3-4607-8ae0-8de6013e1fa4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:45:56.029845 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b641e02c-f6d3-4607-8ae0-8de6013e1fa4 service cinder] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:45:56.040219 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1441/2882] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:45:55 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:46:05.863519 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e0ca8c-4722-4dd4-8c2a-df0926bfd3e4 service cinder] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:05.927965 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e0ca8c-4722-4dd4-8c2a-df0926bfd3e4 service cinder] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:05.932606 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94177507-17bf-4b3a-8757-e51c04b676c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:05.933234 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94177507-17bf-4b3a-8757-e51c04b676c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:05.933510 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94177507-17bf-4b3a-8757-e51c04b676c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:05.933779 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94177507-17bf-4b3a-8757-e51c04b676c2 admin admin] PATH_INFO: `/v3/users/bedf89585ec94eda858e11dd25882810` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:05.935044 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94177507-17bf-4b3a-8757-e51c04b676c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bedf89585ec94eda858e11dd25882810)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:05.936963 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94177507-17bf-4b3a-8757-e51c04b676c2 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:05.982422 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-94177507-17bf-4b3a-8757-e51c04b676c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bedf89585ec94eda858e11dd25882810'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:05.983028 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-94177507-17bf-4b3a-8757-e51c04b676c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bedf89585ec94eda858e11dd25882810'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:05.983620 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-94177507-17bf-4b3a-8757-e51c04b676c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bedf89585ec94eda858e11dd25882810'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:05.996318 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-94177507-17bf-4b3a-8757-e51c04b676c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bedf89585ec94eda858e11dd25882810'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:06.003211 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-94177507-17bf-4b3a-8757-e51c04b676c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bedf89585ec94eda858e11dd25882810'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:06.020159 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1442/2883] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:46:05 2022] DELETE /identity/v3/users/bedf89585ec94eda858e11dd25882810 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:46:06.026003 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef206cbe-17fd-40cf-a248-85b0d0f8f084 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:06.026504 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef206cbe-17fd-40cf-a248-85b0d0f8f084 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:06.026845 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef206cbe-17fd-40cf-a248-85b0d0f8f084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:06.054831 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-ef206cbe-17fd-40cf-a248-85b0d0f8f084 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:06.056592 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-ef206cbe-17fd-40cf-a248-85b0d0f8f084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:06.123763 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1442/2884] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:46:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:46:06.133610 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94177507-17bf-4b3a-8757-e51c04b676c2 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:06.135076 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-94177507-17bf-4b3a-8757-e51c04b676c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:06.168089 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:06.194704 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94177507-17bf-4b3a-8757-e51c04b676c2 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:06.198304 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf3b514-eea8-424b-9047-71e2e7dc7e7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:06.198936 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3b514-eea8-424b-9047-71e2e7dc7e7e service neutron] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:06.199215 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3b514-eea8-424b-9047-71e2e7dc7e7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:06.199490 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3b514-eea8-424b-9047-71e2e7dc7e7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:06.208221 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf3b514-eea8-424b-9047-71e2e7dc7e7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:06.209631 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf3b514-eea8-424b-9047-71e2e7dc7e7e service neutron] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:06.219111 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1443/2885] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:46:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:46:06.482889 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef206cbe-17fd-40cf-a248-85b0d0f8f084 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:06.484418 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-ef206cbe-17fd-40cf-a248-85b0d0f8f084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:06.497929 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef206cbe-17fd-40cf-a248-85b0d0f8f084 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:06.502582 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d126f9b8-1bd8-48cc-8cf3-6b30176c8659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:06.503245 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126f9b8-1bd8-48cc-8cf3-6b30176c8659 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:06.503524 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126f9b8-1bd8-48cc-8cf3-6b30176c8659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:06.503798 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126f9b8-1bd8-48cc-8cf3-6b30176c8659 admin admin] PATH_INFO: `/v3/projects/ba7c5a4890d44dc59a7a7df9a7065a24` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:06.504913 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d126f9b8-1bd8-48cc-8cf3-6b30176c8659 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba7c5a4890d44dc59a7a7df9a7065a24)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:06.507610 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d126f9b8-1bd8-48cc-8cf3-6b30176c8659 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:06.554090 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-d126f9b8-1bd8-48cc-8cf3-6b30176c8659 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba7c5a4890d44dc59a7a7df9a7065a24', 'request_id': 'req-d126f9b8-1bd8-48cc-8cf3-6b30176c8659'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:06.560058 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-d126f9b8-1bd8-48cc-8cf3-6b30176c8659 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba7c5a4890d44dc59a7a7df9a7065a24', 'request_id': 'req-d126f9b8-1bd8-48cc-8cf3-6b30176c8659'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:06.565551 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-d126f9b8-1bd8-48cc-8cf3-6b30176c8659 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba7c5a4890d44dc59a7a7df9a7065a24', 'request_id': 'req-d126f9b8-1bd8-48cc-8cf3-6b30176c8659'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:06.576975 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-d126f9b8-1bd8-48cc-8cf3-6b30176c8659 admin admin] The token cache is being invalidate because project ba7c5a4890d44dc59a7a7df9a7065a24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:46:06.577258 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-d126f9b8-1bd8-48cc-8cf3-6b30176c8659 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:06.577859 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-d126f9b8-1bd8-48cc-8cf3-6b30176c8659 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:06.579337 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1443/2886] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:46:06 2022] DELETE /identity/v3/projects/ba7c5a4890d44dc59a7a7df9a7065a24 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:46:06.583970 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf3b514-eea8-424b-9047-71e2e7dc7e7e service neutron] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:06.593391 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf3b514-eea8-424b-9047-71e2e7dc7e7e service neutron] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:06.648376 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e21d7-f681-4edb-804f-47b0c6f7f929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:06.649070 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e21d7-f681-4edb-804f-47b0c6f7f929 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:06.649349 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e21d7-f681-4edb-804f-47b0c6f7f929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:06.649618 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e21d7-f681-4edb-804f-47b0c6f7f929 admin admin] PATH_INFO: `/v3/users/5b15c48d37f84bb890d2199b42dde8bf` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:06.650752 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6e21d7-f681-4edb-804f-47b0c6f7f929 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b15c48d37f84bb890d2199b42dde8bf)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:06.652669 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6e21d7-f681-4edb-804f-47b0c6f7f929 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:06.696924 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-3c6e21d7-f681-4edb-804f-47b0c6f7f929 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b15c48d37f84bb890d2199b42dde8bf'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:06.697551 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-3c6e21d7-f681-4edb-804f-47b0c6f7f929 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b15c48d37f84bb890d2199b42dde8bf'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:06.698144 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-3c6e21d7-f681-4edb-804f-47b0c6f7f929 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b15c48d37f84bb890d2199b42dde8bf'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:06.710802 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-3c6e21d7-f681-4edb-804f-47b0c6f7f929 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b15c48d37f84bb890d2199b42dde8bf'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:06.718433 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-3c6e21d7-f681-4edb-804f-47b0c6f7f929 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b15c48d37f84bb890d2199b42dde8bf'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:06.737028 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1444/2887] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:46:06 2022] DELETE /identity/v3/users/5b15c48d37f84bb890d2199b42dde8bf => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:46:07.009964 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d126f9b8-1bd8-48cc-8cf3-6b30176c8659 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:07.011434 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-d126f9b8-1bd8-48cc-8cf3-6b30176c8659 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:07.025385 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d126f9b8-1bd8-48cc-8cf3-6b30176c8659 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:07.093838 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34685b96-5e8b-416d-b4a4-d2bf3bb6b713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:07.094762 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34685b96-5e8b-416d-b4a4-d2bf3bb6b713 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:07.094762 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34685b96-5e8b-416d-b4a4-d2bf3bb6b713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:07.095098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34685b96-5e8b-416d-b4a4-d2bf3bb6b713 admin admin] PATH_INFO: `/v3/projects/50242463cdf64de89410353cb261e5aa` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:07.096159 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34685b96-5e8b-416d-b4a4-d2bf3bb6b713 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50242463cdf64de89410353cb261e5aa)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:07.098098 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34685b96-5e8b-416d-b4a4-d2bf3bb6b713 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:07.121440 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:07.145109 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-34685b96-5e8b-416d-b4a4-d2bf3bb6b713 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50242463cdf64de89410353cb261e5aa', 'request_id': 'req-34685b96-5e8b-416d-b4a4-d2bf3bb6b713'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:07.151387 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-34685b96-5e8b-416d-b4a4-d2bf3bb6b713 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50242463cdf64de89410353cb261e5aa', 'request_id': 'req-34685b96-5e8b-416d-b4a4-d2bf3bb6b713'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:07.156908 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-34685b96-5e8b-416d-b4a4-d2bf3bb6b713 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50242463cdf64de89410353cb261e5aa', 'request_id': 'req-34685b96-5e8b-416d-b4a4-d2bf3bb6b713'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:07.168813 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-34685b96-5e8b-416d-b4a4-d2bf3bb6b713 admin admin] The token cache is being invalidate because project 50242463cdf64de89410353cb261e5aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81496) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:46:07.169126 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-34685b96-5e8b-416d-b4a4-d2bf3bb6b713 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:07.169797 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-34685b96-5e8b-416d-b4a4-d2bf3bb6b713 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:07.171333 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1444/2888] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:46:07 2022] DELETE /identity/v3/projects/50242463cdf64de89410353cb261e5aa => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:46:07.176808 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfa490a-1100-4444-90e0-9caf7e6a7587 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:07.177316 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfa490a-1100-4444-90e0-9caf7e6a7587 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:07.177432 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfa490a-1100-4444-90e0-9caf7e6a7587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:07.197049 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-3dfa490a-1100-4444-90e0-9caf7e6a7587 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:07.198243 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-3dfa490a-1100-4444-90e0-9caf7e6a7587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:07.242135 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:07.258004 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1445/2889] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:46:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:46:07.265653 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d877f8-f20a-458e-ad18-257d64e75b8b None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:07.265990 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d877f8-f20a-458e-ad18-257d64e75b8b None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:07.266270 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d877f8-f20a-458e-ad18-257d64e75b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:07.285407 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-87d877f8-f20a-458e-ad18-257d64e75b8b None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:07.286600 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-87d877f8-f20a-458e-ad18-257d64e75b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:07.294715 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1445/2890] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:46:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:46:07.298779 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfa490a-1100-4444-90e0-9caf7e6a7587 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:07.308899 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfa490a-1100-4444-90e0-9caf7e6a7587 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:07.313376 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f2d08d-befc-4032-9308-21c7536177bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:07.313959 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2d08d-befc-4032-9308-21c7536177bd admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:07.314234 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2d08d-befc-4032-9308-21c7536177bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:07.314502 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2d08d-befc-4032-9308-21c7536177bd admin admin] PATH_INFO: `/v3/domains` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:07.315198 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f2d08d-befc-4032-9308-21c7536177bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81497) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:07.315497 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f2d08d-befc-4032-9308-21c7536177bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:07.317828 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f2d08d-befc-4032-9308-21c7536177bd admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:07.329321 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1446/2891] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:46:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 14:46:07.333508 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d877f8-f20a-458e-ad18-257d64e75b8b None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:07.342556 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d877f8-f20a-458e-ad18-257d64e75b8b None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:07.347183 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f1ea56-3348-439a-86c3-ca64ad5d1249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:07.347795 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f1ea56-3348-439a-86c3-ca64ad5d1249 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:07.348073 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f1ea56-3348-439a-86c3-ca64ad5d1249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:07.348341 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f1ea56-3348-439a-86c3-ca64ad5d1249 admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:07.348974 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f1ea56-3348-439a-86c3-ca64ad5d1249 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:07.350887 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f1ea56-3348-439a-86c3-ca64ad5d1249 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:07.379263 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1446/2892] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:46:07 2022] POST /identity/v3/projects => generated 389 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:46:07.383593 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f2d08d-befc-4032-9308-21c7536177bd admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:07.393958 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f2d08d-befc-4032-9308-21c7536177bd admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:07.450357 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8226f42-d63f-4f65-bda0-1575362f32ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:07.450981 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8226f42-d63f-4f65-bda0-1575362f32ca admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:07.451259 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8226f42-d63f-4f65-bda0-1575362f32ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:07.451528 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8226f42-d63f-4f65-bda0-1575362f32ca admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:07.452155 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8226f42-d63f-4f65-bda0-1575362f32ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:07.454035 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8226f42-d63f-4f65-bda0-1575362f32ca admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:07.474014 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1447/2893] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:46:07 2022] POST /identity/v3/users => generated 295 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:46:07.478326 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f1ea56-3348-439a-86c3-ca64ad5d1249 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:07.489249 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f1ea56-3348-439a-86c3-ca64ad5d1249 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:07.493967 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2484a7be-9242-453e-9266-b7cf77fb8898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:07.494559 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484a7be-9242-453e-9266-b7cf77fb8898 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:07.494853 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484a7be-9242-453e-9266-b7cf77fb8898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:07.495127 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484a7be-9242-453e-9266-b7cf77fb8898 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:07.495764 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2484a7be-9242-453e-9266-b7cf77fb8898 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:07.497343 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2484a7be-9242-453e-9266-b7cf77fb8898 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:07.507104 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1447/2894] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:46:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:46:07.510932 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8226f42-d63f-4f65-bda0-1575362f32ca admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:07.520430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8226f42-d63f-4f65-bda0-1575362f32ca admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:07.524959 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965331b4-5c0e-4c49-a67a-207ce6a0fe04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:07.525538 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965331b4-5c0e-4c49-a67a-207ce6a0fe04 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:07.525813 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965331b4-5c0e-4c49-a67a-207ce6a0fe04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:07.526083 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965331b4-5c0e-4c49-a67a-207ce6a0fe04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:07.526709 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965331b4-5c0e-4c49-a67a-207ce6a0fe04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:07.528295 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965331b4-5c0e-4c49-a67a-207ce6a0fe04 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:07.539260 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1448/2895] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:46:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:46:07.544306 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2484a7be-9242-453e-9266-b7cf77fb8898 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:07.554705 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2484a7be-9242-453e-9266-b7cf77fb8898 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:07.559219 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d86ff-0b9b-405c-884f-cc03891f6833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:07.559861 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d86ff-0b9b-405c-884f-cc03891f6833 admin admin] REQUEST_METHOD: `PUT` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:07.560199 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d86ff-0b9b-405c-884f-cc03891f6833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:07.560401 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d86ff-0b9b-405c-884f-cc03891f6833 admin admin] PATH_INFO: `/v3/projects/92e82e3538ec4aec96b54149377482ff/users/2a80074934834eb69e9b204b0a8bda71/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:07.574878 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d86ff-0b9b-405c-884f-cc03891f6833 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92e82e3538ec4aec96b54149377482ff, user_id=2a80074934834eb69e9b204b0a8bda71, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:07.577473 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d86ff-0b9b-405c-884f-cc03891f6833 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:07.589806 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1448/2896] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:46:07 2022] PUT /identity/v3/projects/92e82e3538ec4aec96b54149377482ff/users/2a80074934834eb69e9b204b0a8bda71/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:46:07.601732 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c9ef6-5885-417d-8ae8-ba494285a9d4 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:07.602011 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c9ef6-5885-417d-8ae8-ba494285a9d4 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:07.602309 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c9ef6-5885-417d-8ae8-ba494285a9d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:07.629594 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-496c9ef6-5885-417d-8ae8-ba494285a9d4 None None] MFA Rules not processed for user `2a80074934834eb69e9b204b0a8bda71`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:07.630777 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-496c9ef6-5885-417d-8ae8-ba494285a9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:07.693330 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1449/2897] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:46:07 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:46:07.697638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d86ff-0b9b-405c-884f-cc03891f6833 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:07.759342 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d86ff-0b9b-405c-884f-cc03891f6833 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:07.763882 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12161326-3e5d-4109-8668-89fcfbd807c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:07.764419 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12161326-3e5d-4109-8668-89fcfbd807c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:07.764689 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12161326-3e5d-4109-8668-89fcfbd807c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:07.764954 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12161326-3e5d-4109-8668-89fcfbd807c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:07.765578 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12161326-3e5d-4109-8668-89fcfbd807c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:07.767481 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12161326-3e5d-4109-8668-89fcfbd807c0 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:07.793382 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1449/2898] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:46:07 2022] POST /identity/v3/projects => generated 389 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:46:07.797355 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c9ef6-5885-417d-8ae8-ba494285a9d4 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:07.809968 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c9ef6-5885-417d-8ae8-ba494285a9d4 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:07.854018 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:07.868264 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa0ac5-1105-4f02-bbfb-6434c1f23b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:07.868869 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa0ac5-1105-4f02-bbfb-6434c1f23b2a admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:07.869145 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa0ac5-1105-4f02-bbfb-6434c1f23b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:07.869414 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa0ac5-1105-4f02-bbfb-6434c1f23b2a admin admin] PATH_INFO: `/v3/users` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:07.870100 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fa0ac5-1105-4f02-bbfb-6434c1f23b2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:07.872025 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fa0ac5-1105-4f02-bbfb-6434c1f23b2a admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:07.892418 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1450/2899] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:46:07 2022] POST /identity/v3/users => generated 301 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:46:07.896945 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12161326-3e5d-4109-8668-89fcfbd807c0 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:07.907124 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12161326-3e5d-4109-8668-89fcfbd807c0 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:07.912345 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96407b17-98a5-4178-a47b-0c2e9a83b795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:07.912937 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96407b17-98a5-4178-a47b-0c2e9a83b795 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:07.913215 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96407b17-98a5-4178-a47b-0c2e9a83b795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:07.913759 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96407b17-98a5-4178-a47b-0c2e9a83b795 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:07.914393 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96407b17-98a5-4178-a47b-0c2e9a83b795 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:07.915998 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96407b17-98a5-4178-a47b-0c2e9a83b795 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:07.926337 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1450/2900] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:46:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:46:07.930194 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa0ac5-1105-4f02-bbfb-6434c1f23b2a admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:07.940267 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa0ac5-1105-4f02-bbfb-6434c1f23b2a admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:07.944916 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb3385-0b24-4294-91e6-eb09f8437e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:07.945566 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb3385-0b24-4294-91e6-eb09f8437e7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:07.945848 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb3385-0b24-4294-91e6-eb09f8437e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:07.946123 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb3385-0b24-4294-91e6-eb09f8437e7a admin admin] PATH_INFO: `/v3/projects/c8e3f7088f5345f1b00049c5026929c6/users/75d36e415c5541309205d5ce97547c74/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:07.962615 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fb3385-0b24-4294-91e6-eb09f8437e7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8e3f7088f5345f1b00049c5026929c6, user_id=75d36e415c5541309205d5ce97547c74, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:07.965291 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fb3385-0b24-4294-91e6-eb09f8437e7a admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:07.978765 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1451/2901] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:46:07 2022] PUT /identity/v3/projects/c8e3f7088f5345f1b00049c5026929c6/users/75d36e415c5541309205d5ce97547c74/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:46:07.982761 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96407b17-98a5-4178-a47b-0c2e9a83b795 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:07.992575 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96407b17-98a5-4178-a47b-0c2e9a83b795 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:08.008549 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:08.049475 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61434df7-b7a1-476d-adf0-9e78078b0546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:08.049989 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61434df7-b7a1-476d-adf0-9e78078b0546 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:08.050292 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61434df7-b7a1-476d-adf0-9e78078b0546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:08.050570 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61434df7-b7a1-476d-adf0-9e78078b0546 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:08.051235 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61434df7-b7a1-476d-adf0-9e78078b0546 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:08.052818 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61434df7-b7a1-476d-adf0-9e78078b0546 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:08.062489 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1451/2902] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:46:07 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:46:08.066602 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb3385-0b24-4294-91e6-eb09f8437e7a admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:08.076711 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb3385-0b24-4294-91e6-eb09f8437e7a admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:08.081184 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459e4b8c-f48b-4c43-8b26-6854388db70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:08.082312 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e4b8c-f48b-4c43-8b26-6854388db70f admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:08.082586 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e4b8c-f48b-4c43-8b26-6854388db70f admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:08.082883 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e4b8c-f48b-4c43-8b26-6854388db70f admin admin] PATH_INFO: `/v3/projects/c8e3f7088f5345f1b00049c5026929c6/users/75d36e415c5541309205d5ce97547c74/roles/d00b78c32dbf4cc7925a55bb621d8fcf` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:08.084750 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459e4b8c-f48b-4c43-8b26-6854388db70f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8e3f7088f5345f1b00049c5026929c6, user_id=75d36e415c5541309205d5ce97547c74, role_id=d00b78c32dbf4cc7925a55bb621d8fcf)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:08.087380 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459e4b8c-f48b-4c43-8b26-6854388db70f admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:08.097834 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1452/2903] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:46:08 2022] PUT /identity/v3/projects/c8e3f7088f5345f1b00049c5026929c6/users/75d36e415c5541309205d5ce97547c74/roles/d00b78c32dbf4cc7925a55bb621d8fcf => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:46:08.159182 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9530fa-ed81-449a-aa32-0a0a538c978c None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:08.160326 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9530fa-ed81-449a-aa32-0a0a538c978c None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:08.160432 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9530fa-ed81-449a-aa32-0a0a538c978c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:08.188121 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-fb9530fa-ed81-449a-aa32-0a0a538c978c None None] MFA Rules not processed for user `75d36e415c5541309205d5ce97547c74`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:08.189309 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-fb9530fa-ed81-449a-aa32-0a0a538c978c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:08.256207 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1452/2904] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:46:08 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:46:08.266995 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459e4b8c-f48b-4c43-8b26-6854388db70f admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:08.268727 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-459e4b8c-f48b-4c43-8b26-6854388db70f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:08.325770 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459e4b8c-f48b-4c43-8b26-6854388db70f admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:08.329274 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903d878a-3225-4160-9926-30f5b93e3002 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:08.329862 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d878a-3225-4160-9926-30f5b93e3002 service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:08.330135 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d878a-3225-4160-9926-30f5b93e3002 service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:08.330400 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d878a-3225-4160-9926-30f5b93e3002 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:08.339755 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903d878a-3225-4160-9926-30f5b93e3002 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:08.341161 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903d878a-3225-4160-9926-30f5b93e3002 service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:08.350301 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1453/2905] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:46:08 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:46:08.490333 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9530fa-ed81-449a-aa32-0a0a538c978c None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:08.551471 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9530fa-ed81-449a-aa32-0a0a538c978c None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:08.555913 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac7404-dc44-4d92-b977-aeb18d6a95d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:08.556527 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac7404-dc44-4d92-b977-aeb18d6a95d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:08.556805 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac7404-dc44-4d92-b977-aeb18d6a95d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:08.557076 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac7404-dc44-4d92-b977-aeb18d6a95d2 admin admin] PATH_INFO: `/v3/users/2a80074934834eb69e9b204b0a8bda71` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:08.558179 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac7404-dc44-4d92-b977-aeb18d6a95d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a80074934834eb69e9b204b0a8bda71)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:08.560112 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac7404-dc44-4d92-b977-aeb18d6a95d2 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:08.603414 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:08.606625 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-04ac7404-dc44-4d92-b977-aeb18d6a95d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a80074934834eb69e9b204b0a8bda71'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:08.607431 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-04ac7404-dc44-4d92-b977-aeb18d6a95d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a80074934834eb69e9b204b0a8bda71'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:08.615097 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-04ac7404-dc44-4d92-b977-aeb18d6a95d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a80074934834eb69e9b204b0a8bda71'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:08.615716 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-04ac7404-dc44-4d92-b977-aeb18d6a95d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a80074934834eb69e9b204b0a8bda71'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:08.627626 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-04ac7404-dc44-4d92-b977-aeb18d6a95d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a80074934834eb69e9b204b0a8bda71'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:08.641067 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1453/2906] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:46:08 2022] DELETE /identity/v3/users/2a80074934834eb69e9b204b0a8bda71 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:46:08.645523 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d494ecc-15da-4386-a4f8-e14ade8e7be9 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:08.645808 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d494ecc-15da-4386-a4f8-e14ade8e7be9 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:08.646077 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d494ecc-15da-4386-a4f8-e14ade8e7be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:08.664809 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-7d494ecc-15da-4386-a4f8-e14ade8e7be9 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:08.665980 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-7d494ecc-15da-4386-a4f8-e14ade8e7be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:08.725621 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1454/2907] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:46:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:46:08.735180 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac7404-dc44-4d92-b977-aeb18d6a95d2 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:08.736612 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-04ac7404-dc44-4d92-b977-aeb18d6a95d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:08.793263 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac7404-dc44-4d92-b977-aeb18d6a95d2 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:08.796849 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5281bb9-e764-48b7-892c-70f9fe2580fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:08.797446 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5281bb9-e764-48b7-892c-70f9fe2580fb service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:08.797738 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5281bb9-e764-48b7-892c-70f9fe2580fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:08.797996 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5281bb9-e764-48b7-892c-70f9fe2580fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:08.806837 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5281bb9-e764-48b7-892c-70f9fe2580fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:08.808277 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5281bb9-e764-48b7-892c-70f9fe2580fb service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:08.817572 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1454/2908] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:46:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:46:09.076802 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d494ecc-15da-4386-a4f8-e14ade8e7be9 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:09.078279 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-7d494ecc-15da-4386-a4f8-e14ade8e7be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:09.096950 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d494ecc-15da-4386-a4f8-e14ade8e7be9 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:09.101812 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538987ed-ca19-48b6-a41e-75bd00c85b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:09.102494 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538987ed-ca19-48b6-a41e-75bd00c85b9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:09.102829 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538987ed-ca19-48b6-a41e-75bd00c85b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:09.103244 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538987ed-ca19-48b6-a41e-75bd00c85b9c admin admin] PATH_INFO: `/v3/projects/92e82e3538ec4aec96b54149377482ff` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:09.104287 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538987ed-ca19-48b6-a41e-75bd00c85b9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=92e82e3538ec4aec96b54149377482ff)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:09.106187 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538987ed-ca19-48b6-a41e-75bd00c85b9c admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:09.143856 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:09.156056 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-538987ed-ca19-48b6-a41e-75bd00c85b9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92e82e3538ec4aec96b54149377482ff', 'request_id': 'req-538987ed-ca19-48b6-a41e-75bd00c85b9c'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:09.162213 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-538987ed-ca19-48b6-a41e-75bd00c85b9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92e82e3538ec4aec96b54149377482ff', 'request_id': 'req-538987ed-ca19-48b6-a41e-75bd00c85b9c'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:09.168079 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-538987ed-ca19-48b6-a41e-75bd00c85b9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92e82e3538ec4aec96b54149377482ff', 'request_id': 'req-538987ed-ca19-48b6-a41e-75bd00c85b9c'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:09.180324 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-538987ed-ca19-48b6-a41e-75bd00c85b9c admin admin] The token cache is being invalidate because project 92e82e3538ec4aec96b54149377482ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:46:09.180614 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-538987ed-ca19-48b6-a41e-75bd00c85b9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:09.181240 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-538987ed-ca19-48b6-a41e-75bd00c85b9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:09.182717 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1455/2909] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:46:09 2022] DELETE /identity/v3/projects/92e82e3538ec4aec96b54149377482ff => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:46:09.186638 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5281bb9-e764-48b7-892c-70f9fe2580fb service neutron] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:09.196655 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5281bb9-e764-48b7-892c-70f9fe2580fb service neutron] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:09.252074 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f50bce4-eae0-499e-b178-81ec462d501b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:09.252692 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50bce4-eae0-499e-b178-81ec462d501b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:09.252969 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50bce4-eae0-499e-b178-81ec462d501b admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:09.253762 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50bce4-eae0-499e-b178-81ec462d501b admin admin] PATH_INFO: `/v3/users/75d36e415c5541309205d5ce97547c74` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:09.254900 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f50bce4-eae0-499e-b178-81ec462d501b admin admin] RBAC: Authorizing `identity:delete_user(user_id=75d36e415c5541309205d5ce97547c74)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:09.256817 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f50bce4-eae0-499e-b178-81ec462d501b admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:09.301407 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-4f50bce4-eae0-499e-b178-81ec462d501b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75d36e415c5541309205d5ce97547c74'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:09.302044 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-4f50bce4-eae0-499e-b178-81ec462d501b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75d36e415c5541309205d5ce97547c74'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:09.308870 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-4f50bce4-eae0-499e-b178-81ec462d501b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75d36e415c5541309205d5ce97547c74'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:09.309497 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-4f50bce4-eae0-499e-b178-81ec462d501b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75d36e415c5541309205d5ce97547c74'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:09.316970 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:09.323133 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-4f50bce4-eae0-499e-b178-81ec462d501b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75d36e415c5541309205d5ce97547c74'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:09.336426 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1455/2910] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:46:09 2022] DELETE /identity/v3/users/75d36e415c5541309205d5ce97547c74 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:46:09.607388 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538987ed-ca19-48b6-a41e-75bd00c85b9c admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:09.608850 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-538987ed-ca19-48b6-a41e-75bd00c85b9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:09.623018 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538987ed-ca19-48b6-a41e-75bd00c85b9c admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:09.680001 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b5628-074c-4d9e-9e63-2609ab852807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:09.680572 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b5628-074c-4d9e-9e63-2609ab852807 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:09.681095 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b5628-074c-4d9e-9e63-2609ab852807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:09.681095 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b5628-074c-4d9e-9e63-2609ab852807 admin admin] PATH_INFO: `/v3/projects/c8e3f7088f5345f1b00049c5026929c6` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:09.682641 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b5628-074c-4d9e-9e63-2609ab852807 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8e3f7088f5345f1b00049c5026929c6)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:09.684678 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b5628-074c-4d9e-9e63-2609ab852807 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:09.737228 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-790b5628-074c-4d9e-9e63-2609ab852807 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8e3f7088f5345f1b00049c5026929c6', 'request_id': 'req-790b5628-074c-4d9e-9e63-2609ab852807'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:09.743967 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-790b5628-074c-4d9e-9e63-2609ab852807 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8e3f7088f5345f1b00049c5026929c6', 'request_id': 'req-790b5628-074c-4d9e-9e63-2609ab852807'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:09.749675 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-790b5628-074c-4d9e-9e63-2609ab852807 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8e3f7088f5345f1b00049c5026929c6', 'request_id': 'req-790b5628-074c-4d9e-9e63-2609ab852807'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:09.761578 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-790b5628-074c-4d9e-9e63-2609ab852807 admin admin] The token cache is being invalidate because project c8e3f7088f5345f1b00049c5026929c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:46:09.761854 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-790b5628-074c-4d9e-9e63-2609ab852807 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:09.762472 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-790b5628-074c-4d9e-9e63-2609ab852807 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:09.763974 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1456/2911] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:46:09 2022] DELETE /identity/v3/projects/c8e3f7088f5345f1b00049c5026929c6 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:46:09.768931 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28521875-f1c1-407f-aa54-4f53da54d967 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:09.769289 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28521875-f1c1-407f-aa54-4f53da54d967 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:09.769569 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28521875-f1c1-407f-aa54-4f53da54d967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:09.789322 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-28521875-f1c1-407f-aa54-4f53da54d967 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:09.790505 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-28521875-f1c1-407f-aa54-4f53da54d967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:09.851521 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1456/2912] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:46:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:46:09.859036 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37b993-41ef-42d8-a178-97011fdd3c5d None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:09.859322 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37b993-41ef-42d8-a178-97011fdd3c5d None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:09.859605 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37b993-41ef-42d8-a178-97011fdd3c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:09.880011 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-ca37b993-41ef-42d8-a178-97011fdd3c5d None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:09.881213 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-ca37b993-41ef-42d8-a178-97011fdd3c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:09.889052 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1457/2913] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:46:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:46:09.893121 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28521875-f1c1-407f-aa54-4f53da54d967 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:09.902889 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28521875-f1c1-407f-aa54-4f53da54d967 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:09.907369 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54abcb0-2d94-48d7-ac0a-d081f54450be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:09.907954 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54abcb0-2d94-48d7-ac0a-d081f54450be admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:09.908255 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54abcb0-2d94-48d7-ac0a-d081f54450be admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:09.908520 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54abcb0-2d94-48d7-ac0a-d081f54450be admin admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:09.909160 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54abcb0-2d94-48d7-ac0a-d081f54450be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:09.909458 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54abcb0-2d94-48d7-ac0a-d081f54450be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:09.911589 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54abcb0-2d94-48d7-ac0a-d081f54450be admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:09.923107 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1457/2914] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:46:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:09.927866 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca37b993-41ef-42d8-a178-97011fdd3c5d None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:09.937076 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca37b993-41ef-42d8-a178-97011fdd3c5d None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:09.941700 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0133e648-33c6-4301-8ef3-3087866d9a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:09.942290 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0133e648-33c6-4301-8ef3-3087866d9a7a admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:09.942568 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0133e648-33c6-4301-8ef3-3087866d9a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:09.942867 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0133e648-33c6-4301-8ef3-3087866d9a7a admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:09.943520 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0133e648-33c6-4301-8ef3-3087866d9a7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:09.945402 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0133e648-33c6-4301-8ef3-3087866d9a7a admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:09.971766 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1458/2915] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:46:09 2022] POST /identity/v3/projects => generated 361 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:46:09.976267 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54abcb0-2d94-48d7-ac0a-d081f54450be admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:09.985923 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54abcb0-2d94-48d7-ac0a-d081f54450be admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:10.046732 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea6e743-50a1-46af-bcde-9d2d4462d040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:10.047368 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea6e743-50a1-46af-bcde-9d2d4462d040 admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:10.047645 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea6e743-50a1-46af-bcde-9d2d4462d040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:10.047915 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea6e743-50a1-46af-bcde-9d2d4462d040 admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:10.048535 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea6e743-50a1-46af-bcde-9d2d4462d040 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:10.050461 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea6e743-50a1-46af-bcde-9d2d4462d040 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:10.072517 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1458/2916] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:46:09 2022] POST /identity/v3/users => generated 281 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:46:10.076624 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0133e648-33c6-4301-8ef3-3087866d9a7a admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:10.087110 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0133e648-33c6-4301-8ef3-3087866d9a7a admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:10.091672 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f5fdb-5f0b-445b-a4ef-62ca0aca3c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:10.092260 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f5fdb-5f0b-445b-a4ef-62ca0aca3c00 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:10.092552 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f5fdb-5f0b-445b-a4ef-62ca0aca3c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:10.092805 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f5fdb-5f0b-445b-a4ef-62ca0aca3c00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:10.093434 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f5fdb-5f0b-445b-a4ef-62ca0aca3c00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:10.095149 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f5fdb-5f0b-445b-a4ef-62ca0aca3c00 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:10.105384 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1459/2917] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:46:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:46:10.109169 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea6e743-50a1-46af-bcde-9d2d4462d040 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:10.119005 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea6e743-50a1-46af-bcde-9d2d4462d040 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:10.123486 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33174e73-e5d7-4c11-a531-19eabbc504a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:10.124071 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33174e73-e5d7-4c11-a531-19eabbc504a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:10.124348 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33174e73-e5d7-4c11-a531-19eabbc504a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:10.124613 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33174e73-e5d7-4c11-a531-19eabbc504a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:10.125237 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33174e73-e5d7-4c11-a531-19eabbc504a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:10.126865 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33174e73-e5d7-4c11-a531-19eabbc504a2 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:10.137135 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1459/2918] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:46:10 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:46:10.141219 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f5fdb-5f0b-445b-a4ef-62ca0aca3c00 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:10.151445 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f5fdb-5f0b-445b-a4ef-62ca0aca3c00 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:10.155947 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18830743-660a-4fca-a548-7577a8e875fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:10.156581 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18830743-660a-4fca-a548-7577a8e875fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:10.156854 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18830743-660a-4fca-a548-7577a8e875fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:10.157123 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18830743-660a-4fca-a548-7577a8e875fa admin admin] PATH_INFO: `/v3/projects/cce47923fe5b4d28aa13f2398c627fef/users/407877ff6de14e28ad540704345da812/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:10.169815 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:10.172873 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18830743-660a-4fca-a548-7577a8e875fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=cce47923fe5b4d28aa13f2398c627fef, user_id=407877ff6de14e28ad540704345da812, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:10.175693 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18830743-660a-4fca-a548-7577a8e875fa admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:10.189601 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1460/2919] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:46:10 2022] PUT /identity/v3/projects/cce47923fe5b4d28aa13f2398c627fef/users/407877ff6de14e28ad540704345da812/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 14:46:10.200631 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79125904-c7e0-4fc6-8371-dfd3b0212628 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:10.201126 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79125904-c7e0-4fc6-8371-dfd3b0212628 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:10.201244 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79125904-c7e0-4fc6-8371-dfd3b0212628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:10.228830 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-79125904-c7e0-4fc6-8371-dfd3b0212628 None None] MFA Rules not processed for user `407877ff6de14e28ad540704345da812`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:10.230015 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-79125904-c7e0-4fc6-8371-dfd3b0212628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:10.265982 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:10.293730 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1460/2920] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:46:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:46:10.326397 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18830743-660a-4fca-a548-7577a8e875fa admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:10.328123 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-18830743-660a-4fca-a548-7577a8e875fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:10.383291 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18830743-660a-4fca-a548-7577a8e875fa admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:10.386920 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa0a0a9-419d-4b4b-82de-d2361edd33e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:10.387517 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa0a0a9-419d-4b4b-82de-d2361edd33e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:10.387793 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa0a0a9-419d-4b4b-82de-d2361edd33e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:10.388064 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa0a0a9-419d-4b4b-82de-d2361edd33e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:10.396582 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa0a0a9-419d-4b4b-82de-d2361edd33e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:10.398033 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa0a0a9-419d-4b4b-82de-d2361edd33e5 service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:10.406608 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1461/2921] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:46:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:46:10.422789 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79125904-c7e0-4fc6-8371-dfd3b0212628 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:10.483947 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79125904-c7e0-4fc6-8371-dfd3b0212628 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:10.488393 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a4042-20f1-49aa-97db-b6085704b28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:10.489052 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a4042-20f1-49aa-97db-b6085704b28d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:10.489324 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a4042-20f1-49aa-97db-b6085704b28d admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:10.489588 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a4042-20f1-49aa-97db-b6085704b28d admin admin] PATH_INFO: `/v3/users/407877ff6de14e28ad540704345da812` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:10.490696 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616a4042-20f1-49aa-97db-b6085704b28d admin admin] RBAC: Authorizing `identity:delete_user(user_id=407877ff6de14e28ad540704345da812)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:10.492637 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616a4042-20f1-49aa-97db-b6085704b28d admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:10.537203 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-616a4042-20f1-49aa-97db-b6085704b28d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '407877ff6de14e28ad540704345da812'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:10.537867 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-616a4042-20f1-49aa-97db-b6085704b28d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '407877ff6de14e28ad540704345da812'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:10.545252 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-616a4042-20f1-49aa-97db-b6085704b28d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '407877ff6de14e28ad540704345da812'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:10.545888 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-616a4042-20f1-49aa-97db-b6085704b28d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '407877ff6de14e28ad540704345da812'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:10.557895 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-616a4042-20f1-49aa-97db-b6085704b28d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '407877ff6de14e28ad540704345da812'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:10.571482 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1461/2922] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:46:10 2022] DELETE /identity/v3/users/407877ff6de14e28ad540704345da812 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:46:10.576108 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e75308b-df2b-41bb-99a8-d7d69167a31e None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:10.576397 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e75308b-df2b-41bb-99a8-d7d69167a31e None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:10.576674 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e75308b-df2b-41bb-99a8-d7d69167a31e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:10.595873 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-9e75308b-df2b-41bb-99a8-d7d69167a31e None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:10.597062 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-9e75308b-df2b-41bb-99a8-d7d69167a31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:10.654537 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1462/2923] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:46:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:46:10.664718 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a4042-20f1-49aa-97db-b6085704b28d admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:10.666138 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-616a4042-20f1-49aa-97db-b6085704b28d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:10.722604 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a4042-20f1-49aa-97db-b6085704b28d admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:10.726211 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a0edd1-5301-4f31-a6da-b5e255fb1cb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:10.726827 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0edd1-5301-4f31-a6da-b5e255fb1cb5 service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:10.727114 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0edd1-5301-4f31-a6da-b5e255fb1cb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:10.727384 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0edd1-5301-4f31-a6da-b5e255fb1cb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:10.736267 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a0edd1-5301-4f31-a6da-b5e255fb1cb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:10.737648 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a0edd1-5301-4f31-a6da-b5e255fb1cb5 service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:10.742477 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:10.747658 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1462/2924] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:46:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:46:11.012432 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e75308b-df2b-41bb-99a8-d7d69167a31e None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:11.013866 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-9e75308b-df2b-41bb-99a8-d7d69167a31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:11.025988 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e75308b-df2b-41bb-99a8-d7d69167a31e None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:11.030614 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d460d-86bb-4eb0-a0e5-4708d3a7550d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:11.031275 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d460d-86bb-4eb0-a0e5-4708d3a7550d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:11.031554 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d460d-86bb-4eb0-a0e5-4708d3a7550d admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:11.031827 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d460d-86bb-4eb0-a0e5-4708d3a7550d admin admin] PATH_INFO: `/v3/projects/cce47923fe5b4d28aa13f2398c627fef` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:11.032941 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109d460d-86bb-4eb0-a0e5-4708d3a7550d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cce47923fe5b4d28aa13f2398c627fef)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:11.034834 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109d460d-86bb-4eb0-a0e5-4708d3a7550d admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:11.081430 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-109d460d-86bb-4eb0-a0e5-4708d3a7550d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cce47923fe5b4d28aa13f2398c627fef', 'request_id': 'req-109d460d-86bb-4eb0-a0e5-4708d3a7550d'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:11.087960 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-109d460d-86bb-4eb0-a0e5-4708d3a7550d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cce47923fe5b4d28aa13f2398c627fef', 'request_id': 'req-109d460d-86bb-4eb0-a0e5-4708d3a7550d'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:11.093397 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-109d460d-86bb-4eb0-a0e5-4708d3a7550d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cce47923fe5b4d28aa13f2398c627fef', 'request_id': 'req-109d460d-86bb-4eb0-a0e5-4708d3a7550d'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:11.106177 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-109d460d-86bb-4eb0-a0e5-4708d3a7550d admin admin] The token cache is being invalidate because project cce47923fe5b4d28aa13f2398c627fef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:46:11.106458 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-109d460d-86bb-4eb0-a0e5-4708d3a7550d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:11.107077 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-109d460d-86bb-4eb0-a0e5-4708d3a7550d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:11.108554 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1463/2925] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:46:11 2022] DELETE /identity/v3/projects/cce47923fe5b4d28aa13f2398c627fef => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:46:11.113476 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e13f43-d9c2-410d-9c77-91205b1e0690 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:11.113799 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e13f43-d9c2-410d-9c77-91205b1e0690 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:11.114075 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e13f43-d9c2-410d-9c77-91205b1e0690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:11.133099 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-f7e13f43-d9c2-410d-9c77-91205b1e0690 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:11.134286 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-f7e13f43-d9c2-410d-9c77-91205b1e0690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:11.193731 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1463/2926] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:46:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 14:46:11.201509 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3043f6b8-0b7e-4b9b-85ec-8fbedcc0d714 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:11.201790 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3043f6b8-0b7e-4b9b-85ec-8fbedcc0d714 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:11.202064 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3043f6b8-0b7e-4b9b-85ec-8fbedcc0d714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:11.222318 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-3043f6b8-0b7e-4b9b-85ec-8fbedcc0d714 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:11.223597 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-3043f6b8-0b7e-4b9b-85ec-8fbedcc0d714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:11.231532 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1464/2927] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:46:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:46:11.235663 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e13f43-d9c2-410d-9c77-91205b1e0690 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:11.245172 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e13f43-d9c2-410d-9c77-91205b1e0690 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:11.249759 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8787423-218e-4b64-8a3f-c33e3a205c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:11.250360 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8787423-218e-4b64-8a3f-c33e3a205c06 admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:11.250639 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8787423-218e-4b64-8a3f-c33e3a205c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:11.250939 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8787423-218e-4b64-8a3f-c33e3a205c06 admin admin] PATH_INFO: `/v3/domains` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:11.251583 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8787423-218e-4b64-8a3f-c33e3a205c06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81496) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 14:46:11.251883 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8787423-218e-4b64-8a3f-c33e3a205c06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:11.253494 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8787423-218e-4b64-8a3f-c33e3a205c06 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:11.265637 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1464/2928] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 14:46:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 14:46:11.270334 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3043f6b8-0b7e-4b9b-85ec-8fbedcc0d714 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:11.279391 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3043f6b8-0b7e-4b9b-85ec-8fbedcc0d714 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:11.283958 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283a55a-8c63-4121-821a-fccc6f7181d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:11.284546 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283a55a-8c63-4121-821a-fccc6f7181d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:11.284820 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283a55a-8c63-4121-821a-fccc6f7181d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:11.285086 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283a55a-8c63-4121-821a-fccc6f7181d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:11.285712 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b283a55a-8c63-4121-821a-fccc6f7181d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:11.287613 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b283a55a-8c63-4121-821a-fccc6f7181d3 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:11.297126 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:11.316011 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1465/2929] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 14:46:11 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:46:11.320793 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8787423-218e-4b64-8a3f-c33e3a205c06 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:11.330489 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8787423-218e-4b64-8a3f-c33e3a205c06 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:11.385351 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d04601-481b-4293-b9ea-2b3423e8c72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:11.385941 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d04601-481b-4293-b9ea-2b3423e8c72f admin admin] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:11.386213 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d04601-481b-4293-b9ea-2b3423e8c72f admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:11.386480 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d04601-481b-4293-b9ea-2b3423e8c72f admin admin] PATH_INFO: `/v3/users` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:11.387121 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d04601-481b-4293-b9ea-2b3423e8c72f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:11.389032 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d04601-481b-4293-b9ea-2b3423e8c72f admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:11.409514 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1465/2930] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 14:46:11 2022] POST /identity/v3/users => generated 283 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 14:46:11.414428 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283a55a-8c63-4121-821a-fccc6f7181d3 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:11.424484 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283a55a-8c63-4121-821a-fccc6f7181d3 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:11.429583 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b9d52-063e-445b-8041-31e482149671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:11.430168 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b9d52-063e-445b-8041-31e482149671 admin admin] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:11.430442 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b9d52-063e-445b-8041-31e482149671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:11.430707 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b9d52-063e-445b-8041-31e482149671 admin admin] PATH_INFO: `/v3/roles` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:11.431371 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4b9d52-063e-445b-8041-31e482149671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:11.432965 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4b9d52-063e-445b-8041-31e482149671 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:11.442521 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1466/2931] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:46:11.446561 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d04601-481b-4293-b9ea-2b3423e8c72f admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:11.456350 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d04601-481b-4293-b9ea-2b3423e8c72f admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:11.461188 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f51859-6d92-4e0f-9a66-0936e20052af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:11.461769 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f51859-6d92-4e0f-9a66-0936e20052af admin admin] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:11.462040 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f51859-6d92-4e0f-9a66-0936e20052af admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:11.462307 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f51859-6d92-4e0f-9a66-0936e20052af admin admin] PATH_INFO: `/v3/roles` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:11.462972 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f51859-6d92-4e0f-9a66-0936e20052af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:11.464538 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f51859-6d92-4e0f-9a66-0936e20052af admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:11.475332 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1466/2932] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 14:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 14:46:11.479203 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b9d52-063e-445b-8041-31e482149671 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:11.489388 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b9d52-063e-445b-8041-31e482149671 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:11.493872 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d487812-45c0-4e93-b488-1dabf3b47982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:11.494497 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d487812-45c0-4e93-b488-1dabf3b47982 admin admin] REQUEST_METHOD: `PUT` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:11.494772 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d487812-45c0-4e93-b488-1dabf3b47982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:11.495074 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d487812-45c0-4e93-b488-1dabf3b47982 admin admin] PATH_INFO: `/v3/projects/ffede372c51c45f88c7d30d39911794f/users/9944919215df4634b7fb9d99ccf24a99/roles/640af5182e6848498c16a19cddbfb738` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:11.509128 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d487812-45c0-4e93-b488-1dabf3b47982 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffede372c51c45f88c7d30d39911794f, user_id=9944919215df4634b7fb9d99ccf24a99, role_id=640af5182e6848498c16a19cddbfb738)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:11.511770 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d487812-45c0-4e93-b488-1dabf3b47982 admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:11.524272 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1467/2933] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 14:46:11 2022] PUT /identity/v3/projects/ffede372c51c45f88c7d30d39911794f/users/9944919215df4634b7fb9d99ccf24a99/roles/640af5182e6848498c16a19cddbfb738 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:46:11.534135 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4b955-1e93-44f4-aee4-8d7da124dc81 None None] REQUEST_METHOD: `POST` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:11.534441 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4b955-1e93-44f4-aee4-8d7da124dc81 None None] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:11.534712 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4b955-1e93-44f4-aee4-8d7da124dc81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:11.563049 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.auth.core [None req-c5b4b955-1e93-44f4-aee4-8d7da124dc81 None None] MFA Rules not processed for user `9944919215df4634b7fb9d99ccf24a99`. Rule list: `[]` (Enabled: `True`). {{(pid=81496) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:11.564231 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-c5b4b955-1e93-44f4-aee4-8d7da124dc81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:11.617871 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:11.627719 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1467/2934] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:46:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:46:11.639213 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d487812-45c0-4e93-b488-1dabf3b47982 admin admin] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:11.640885 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-1d487812-45c0-4e93-b488-1dabf3b47982 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:11.696083 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d487812-45c0-4e93-b488-1dabf3b47982 admin admin] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:11.699647 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece6f8a-9ca8-4bcd-969e-cd356c0d5a78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63921623056045a9b97b2de3ca418138', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:11.700242 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece6f8a-9ca8-4bcd-969e-cd356c0d5a78 service cinder] REQUEST_METHOD: `GET` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:11.700516 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece6f8a-9ca8-4bcd-969e-cd356c0d5a78 service cinder] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:11.700786 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece6f8a-9ca8-4bcd-969e-cd356c0d5a78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:11.709349 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eece6f8a-9ca8-4bcd-969e-cd356c0d5a78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:11.710761 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eece6f8a-9ca8-4bcd-969e-cd356c0d5a78 service cinder] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:11.719950 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1468/2935] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:46:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:46:21.628068 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=2 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:21.715368 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=45,keystone:INSERT=2 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:46:54.177920 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4b955-1e93-44f4-aee4-8d7da124dc81 None None] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:54.239205 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4b955-1e93-44f4-aee4-8d7da124dc81 None None] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:54.243564 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496266ec-96c5-44c5-8a0e-1a3c2e025b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:54.244179 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496266ec-96c5-44c5-8a0e-1a3c2e025b73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:54.244453 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496266ec-96c5-44c5-8a0e-1a3c2e025b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:54.244720 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496266ec-96c5-44c5-8a0e-1a3c2e025b73 admin admin] PATH_INFO: `/v3/users/9944919215df4634b7fb9d99ccf24a99` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:54.245832 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496266ec-96c5-44c5-8a0e-1a3c2e025b73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9944919215df4634b7fb9d99ccf24a99)` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:54.247799 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496266ec-96c5-44c5-8a0e-1a3c2e025b73 admin admin] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:54.292990 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-496266ec-96c5-44c5-8a0e-1a3c2e025b73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9944919215df4634b7fb9d99ccf24a99'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:54.293663 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-496266ec-96c5-44c5-8a0e-1a3c2e025b73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9944919215df4634b7fb9d99ccf24a99'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:54.300770 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-496266ec-96c5-44c5-8a0e-1a3c2e025b73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9944919215df4634b7fb9d99ccf24a99'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:54.301394 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-496266ec-96c5-44c5-8a0e-1a3c2e025b73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9944919215df4634b7fb9d99ccf24a99'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:54.313005 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.notifications [None req-496266ec-96c5-44c5-8a0e-1a3c2e025b73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9944919215df4634b7fb9d99ccf24a99'} {{(pid=81496) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:54.325504 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1468/2936] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 14:46:54 2022] DELETE /identity/v3/users/9944919215df4634b7fb9d99ccf24a99 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:46:54.330070 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3ba76-d427-4397-b955-36a3758b4ba5 None None] REQUEST_METHOD: `POST` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:54.330578 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3ba76-d427-4397-b955-36a3758b4ba5 None None] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:54.330691 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3ba76-d427-4397-b955-36a3758b4ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:54.350118 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.auth.core [None req-bba3ba76-d427-4397-b955-36a3758b4ba5 None None] MFA Rules not processed for user `a6fda065d398415a9b8308457ab0f0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81497) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 14:46:54.351332 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-bba3ba76-d427-4397-b955-36a3758b4ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:54.407695 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1469/2937] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 14:46:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 14:46:54.418089 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496266ec-96c5-44c5-8a0e-1a3c2e025b73 admin admin] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:54.419514 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-496266ec-96c5-44c5-8a0e-1a3c2e025b73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:54.475491 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496266ec-96c5-44c5-8a0e-1a3c2e025b73 admin admin] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:54.479119 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c8ff73-d7e8-4717-8de4-5fbf8591b8f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9348164132846d2b593ddfa29a820eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:54.479717 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8ff73-d7e8-4717-8de4-5fbf8591b8f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:54.479995 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8ff73-d7e8-4717-8de4-5fbf8591b8f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:54.480267 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8ff73-d7e8-4717-8de4-5fbf8591b8f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:54.489001 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c8ff73-d7e8-4717-8de4-5fbf8591b8f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:54.490499 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c8ff73-d7e8-4717-8de4-5fbf8591b8f4 service neutron] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:54.499761 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1469/2938] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:46:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:46:54.754926 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3ba76-d427-4397-b955-36a3758b4ba5 None None] Authenticating user token {{(pid=81497) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:46:54.756477 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.fernet_utils [None req-bba3ba76-d427-4397-b955-36a3758b4ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81497) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:46:54.768815 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3ba76-d427-4397-b955-36a3758b4ba5 None None] Validating token access rules against request {{(pid=81497) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:46:54.775171 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1d340a-d46a-462b-a443-e70e2fd39eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6fda065d398415a9b8308457ab0f0dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f8134960517435ea813cbf2d41e25d7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81497) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:46:54.775802 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d340a-d46a-462b-a443-e70e2fd39eee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:46:54.776124 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d340a-d46a-462b-a443-e70e2fd39eee admin admin] SCRIPT_NAME: `/identity` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:46:54.776347 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d340a-d46a-462b-a443-e70e2fd39eee admin admin] PATH_INFO: `/v3/projects/ffede372c51c45f88c7d30d39911794f` {{(pid=81497) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:46:54.777466 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1d340a-d46a-462b-a443-e70e2fd39eee admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffede372c51c45f88c7d30d39911794f)` {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:46:54.780615 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1d340a-d46a-462b-a443-e70e2fd39eee admin admin] RBAC: Authorization granted {{(pid=81497) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:46:54.828079 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-2b1d340a-d46a-462b-a443-e70e2fd39eee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffede372c51c45f88c7d30d39911794f', 'request_id': 'req-2b1d340a-d46a-462b-a443-e70e2fd39eee'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:54.836193 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-2b1d340a-d46a-462b-a443-e70e2fd39eee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffede372c51c45f88c7d30d39911794f', 'request_id': 'req-2b1d340a-d46a-462b-a443-e70e2fd39eee'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:54.841099 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-2b1d340a-d46a-462b-a443-e70e2fd39eee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffede372c51c45f88c7d30d39911794f', 'request_id': 'req-2b1d340a-d46a-462b-a443-e70e2fd39eee'} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:54.856710 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-2b1d340a-d46a-462b-a443-e70e2fd39eee admin admin] The token cache is being invalidate because project ffede372c51c45f88c7d30d39911794f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81497) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 14:46:54.857184 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-2b1d340a-d46a-462b-a443-e70e2fd39eee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:54.857993 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG keystone.notifications [None req-2b1d340a-d46a-462b-a443-e70e2fd39eee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81497) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 14:46:54.859549 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: [pid: 81497|app: 0|req: 1470/2939] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 14:46:54 2022] DELETE /identity/v3/projects/ffede372c51c45f88c7d30d39911794f => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 14:47:04.495749 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:47:04.851201 dv-cinder-859193-1-iscsi devstack@keystone.service[81497]: DEBUG dbcounter [-] [81497] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81497) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 14:50:34.707569 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c8ff73-d7e8-4717-8de4-5fbf8591b8f4 service neutron] Authenticating user token {{(pid=81496) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 14:50:34.708953 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-06c8ff73-d7e8-4717-8de4-5fbf8591b8f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:50:34.779987 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c8ff73-d7e8-4717-8de4-5fbf8591b8f4 service neutron] Validating token access rules against request {{(pid=81496) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 14:50:34.839240 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480d844-8918-48d8-92cf-cf8db31c6b0f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18fe384ecd454fb298dcc788f41dd44b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36a3ae7bab25471fb989136150b6438c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81496) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 14:50:34.839849 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480d844-8918-48d8-92cf-cf8db31c6b0f service placement] REQUEST_METHOD: `GET` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 14:50:34.840126 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480d844-8918-48d8-92cf-cf8db31c6b0f service placement] SCRIPT_NAME: `/identity` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 14:50:34.840396 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480d844-8918-48d8-92cf-cf8db31c6b0f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81496) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 14:50:34.842151 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.fernet_utils [None req-8480d844-8918-48d8-92cf-cf8db31c6b0f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81496) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 14:50:34.853211 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8480d844-8918-48d8-92cf-cf8db31c6b0f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 14:50:34.854784 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8480d844-8918-48d8-92cf-cf8db31c6b0f service placement] RBAC: Authorization granted {{(pid=81496) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 14:50:34.865040 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: [pid: 81496|app: 0|req: 1470/2940] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 14:50:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 14:50:44.860792 dv-cinder-859193-1-iscsi devstack@keystone.service[81496]: DEBUG dbcounter [-] [81496] Writing DB stats keystone:SELECT=58 {{(pid=81496) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}